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PREFACE 


The  military  is  formulating  new  visions,  strategies,  and  concepts  that 
capitalize  on  emerging  information- age  technologies  to  provide  its 
warfighters  with  significantly  improved  capabilities  to  meet  the 
national  security  challenges  of  the  21st  century.  These  programs  are 
described  in  such  documents  as  the  Quadrennial  Defense  Review, 
Joint  Vision  2020,  a  variety  of  publications  describing  network¬ 
centric  warfare  (NCW),  and  other  documents  describing  military 
transformation.  Joint  Vision  2020  provides  an  important  starting 
point  for  describing  a  future  warfighting  concept  that  has  since 
evolved  into  NCW.  A  key  tenet  of  Joint  Vision  2020  is  that  information 
superiority  will  enable  decision  dominance,  new  Joint  operational 
concepts,  and  a  decisive  advantage  over  future  adversaries.  To  create 
and  leverage  information  superiority,  it  is  foreseen  that,  under  some 
circumstances,  a  mix  of  command,  control,  communications,  com¬ 
puters,  intelligence,  surveillance,  and  reconnaissance  (C4ISR)  capa¬ 
bilities  would  interoperate  with  weapon  systems  and  forces  on  an 
end-to-end  basis  through  a  network- centric  information  environ¬ 
ment  to  achieve  significant  improvements  in  awareness,  shared 
awareness,  and  synchronization.  The  military  is  embarked  on  a 
series  of  analyses  and  experiments  to  improve  its  understanding  of 
the  potential  of  these  NCW  concepts. 

The  Assistant  Secretary  of  Defense  for  Networks  and  Information 
Integration  (ASD  Nil),  through  the  Command  and  Control  Research 
Program,  asked  RAND  to  help  develop  methods  and  tools  that  could 
improve  the  assessment  of  C4ISR  capabilities  and  processes  to  the 
achievement  of  NCW  concepts,  including  awareness,  shared  aware¬ 
ness,  and  synchronization.  In  response  to  this  request,  the  RAND 
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Corporation  has  been  participating  in  the  Information  Superiority 
Metrics  Working  Group,  under  the  auspices  of  ASD  NIL  The  group’s 
purpose  is  to  describe  key  concepts  and  related  metrics  that  are  nec¬ 
essary  to  explore  part  of  the  proposed  NCW  value  chain — from 
information  quality  through  awareness,  shared  awareness,  collabo¬ 
ration,  and  synchronization,  to  force  effectiveness  and  mission  out¬ 
come.  This  report  presents  a  methodology — including  metrics,  for¬ 
mulas  for  generating  metrics,  and  transfer  functions  for  generating 
dependencies  between  metrics — for  measuring  the  quality  of  infor¬ 
mation  and  its  influence  on  the  degree  of  shared  situational  aware¬ 
ness. 

This  research  was  conducted  within  the  Acquisition  and  Technology 
Policy  Center  of  RAND’s  National  Defense  Research  Institute  (NDRI). 
NDRI  is  a  federally  funded  research  and  development  center  spon¬ 
sored  by  the  Office  of  the  Secretary  of  Defense,  the  Joint  Staff,  the 
unified  commands,  and  the  defense  agencies. 
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SUMMARY 


The  military  is  formulating  new  visions,  strategies,  and  concepts  that 
capitalize  on  emerging  information- age  technologies  to  provide  its 
warfighters  with  significantly  improved  capabilities  to  meet  the 
national  security  challenges  of  the  21st  century.  New,  networked 
C4ISR  capabilities  promise  information  superiority  and  decision 
dominance  that  will  enhance  the  quality  and  speed  of  command  and 
enable  revolutionary  warfighting  concepts.  Assessing  the  contribu¬ 
tion  of  C4ISR  toward  achieving  an  NCW  capability  is  a  major  chal¬ 
lenge  for  the  Department  of  Defense  (DoD).  Much  like  the  develop¬ 
ment  of  a  new  branch  of  science,  this  requires  defining  concepts, 
metrics,  hypotheses,  and  analytical  methodologies  that  can  be  used 
to  focus  research  efforts,  identify  and  compare  alternatives,  and 
measure  progress. 

INTRODUCTION 

An  important  first  step  is  to  improve  our  understanding  of  how 
improved  C4ISR  capabilities  and  related  changes  in  command  con¬ 
trol  processes  contribute  to  the  achievement  of  core  information- 
superiority  concepts,  such  as  situational  awareness,  shared  situa¬ 
tional  awareness,  and  synchronization.  Establishing  a  quantifiable 
link  between  improved  C4ISR  capabilities  and  combat  outcomes  has 
been  extremely  elusive  and  is  therefore  a  major  challenge.  In  this 
work,  therefore,  we  develop  a  mathematical  framework  that  can 
facilitate  the  development  of  alternative  measures  of  performance 
and  associated  metrics  that  assess  the  contribution  of  information 
quality  and  team  collaboration  on  shared  situational  awareness.  The 
emphasis  is  on  the  development  of  the  framework. 


xiii 


xiv  Exploring  Information  Superiority 


The  research  reported  here  builds  on  the  work  of  the  ASD  Nil  Infor¬ 
mation  Superiority  Metrics  Working  Group.  This  body  has  developed 
working  definitions,  specific  characteristics  and  attributes  of  key 
concepts,  and  the  relationships  among  them  that  are  needed  to  mea¬ 
sure  the  degree  to  which  information- superiority  concepts  are  real¬ 
ized  and  their  influence  on  the  conduct  and  effectiveness  of  military 
operations.  The  research  is  also  consistent  with  the  NCW  Conceptual 
Framework,  which  DoD’s  Office  of  Force  Transformation  and  ASD 
Nil  are  developing  jointly.  The  NCW  Conceptual  Framework  is  an 
assessment  tool  that  includes  measures,  general  forms  for  metrics, 
and  relationships  between  the  measures  and  metrics.  It  contains  a 
large  number  of  measures  related  to  the  complete  array  of  concepts 
associated  with  NCW,  ranging  from  networking  hardware  through 
decisionmaking  capabilities  and  synchronization  of  actions.  The 
group’s  metrics,  and  this  report’s  scope,  are  largely  limited  to  the 
information  and  awareness  components  of  the  NCW  Conceptual 
Framework,  and  explore  these  components  in  more  detail  than  does 
the  framework. 

We  begin  by  defining  a  reference  model  for  discussing  such  issues  in 
terms  of  three  domains:  that  of  ground  truth  (the  physical  domain); 
that  of  sensed  information  (the  information  domain);  and  that  in 
which  individual  situational  awareness,  shared  situational  aware¬ 
ness,  collaboration,  and  decisionmaking  occur  (the  cognitive  do¬ 
main).  The  C4ISR  process  is  seen  as  extracting  data  from  ground 
truth  and  processing  the  data  in  the  information  domain  to  produce 
a  common  relevant  operating  picture  (CROP).  The  quality  of  the 
CROP  and  the  quality  of  team  collaboration  combine  to  heighten  (or 
degrade)  shared  situational  awareness  in  the  cognitive  domain. 

THE  ANALYTIC  FRAMEWORK 

The  objective  of  this  research  is  to  develop  a  quantitative  methodol¬ 
ogy  that  allows  us  to  link  improvements  in  C4ISR  capabilities  to  their 
effects  on  combat  outcomes.  For  this  first  effort,  we  have  confined 
our  work  to  assessing  the  effects  of  data- collection  and  information- 
fusion  processes,  and  the  dissemination  of  the  fused  CROP  on  indi¬ 
vidual  situational  awareness  and,  through  the  collaboration  process, 
on  shared  situational  awareness. 
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Figure  S.l,  the  C4ISR  Information  Superiority  Reference  Model, 
describes  the  activities  associated  with  the  above  processes.  This 
model  envisions  the  three  “domains”  extending  from  the  battlefield 
environment  to  cognitive  awareness  of  the  battlefield  situation  and 
decision. 

This  report  uses  a  generic  C4ISR  architecture  to  build  a  model  repre¬ 
senting  the  contributions  of  these  processes.  The  architecture  can  be 
thought  of  as  a  six-stage  process  that  comprises  the  following: 

0.  acceptance  of  the  existence  of  physical  ground  truth,  restricted 
here  to  physical  battlespace  entities  and  their  attributes  (the  ini¬ 
tial  state) 

1.  sensing  of  ground  truth  by  an  array  of  network  sensors 

2.  fusion  of  sensor  data  by  a  centralized  set  of  fusion  facilities 

3.  distribution  of  resulting  information  (the  CROP)  to  the  users  over 
a  potentially  noisy  and  unreliable  network 

RAND  MR1467-S.1 


Cognitive  domain 


NOTE:  The  activities  depicted  in  each  of  the  domain  “boxes”  may  not  be  complete. 
We  focus  on  those  activities  pertinent  to  our  research. 


Figure  S.l — The  Information  Superiority  Reference  Model 
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4.  individual  interpretation  of  the  CROP,  with  the  quality  of  the 
interpretation  depending  on  the  user’s  skills  and  abilities 

5.  collaboration  to  improve  interpretation  of  the  CROP,  with  the 
quality  of  the  interpretation  based  on  individual  and  group  char¬ 
acteristics. 

The  value  of  the  collection,  fusion,  dissemination,  interpretation, 
and  collaboration  processes  to  combat  operations  within  the  above 
generic  C4ISR  architecture  is  described  through  the  several  transfor¬ 
mation  functions,  as  shown  in  Figure  S.2.  The  development  of  a 
quantitative  framework  is  based  on  these  transformations. 

Enemy  battlefield  entities  (units  and  weapon  systems)  are  described 
in  terms  of  their  features  or  characteristics;  hence,  the  quality  of  the 
information  concerning  the  entities  is  an  assessment  of  how  well  the 
C4ISR  system  estimates  the  features  of  the  collected  set  of  enemy 
units  in  the  battlespace.  A  conditional  product  form  model  is  used  to 
measure  the  effects  of  the  NCW  value  chain  transformations  on  the 
information- domain  measures  (quality  of  sensor  information,  qual¬ 
ity  of  CROP,  quality  of  shared  CROP),  and  a  more-general  functional 
model  measures  the  effects  on  the  cognitive  domain  measures 
(situational  awareness,  shared  situational  awareness) . 

THE  PHYSICAL  AND  INFORMATION  DOMAINS 

We  applied  the  methodology  to  the  measures  in  the  physical  and 
information  domains.  The  feature  matrix,  F  =  [F1(  F2, . . . ,  Fm],  is  a  set 
of  vectors,  F,,  each  of  which  represents  the  relevant  physical  charac¬ 
teristics  of  the  enemy.  In  the  physical  domain,  F0  is  a  feature  matrix 
representing  the  physical  ground  truth  features  of  all  enemy  units. 

Sensor  Metrics 

Using  F0  as  an  input,  we  first  developed  metrics  formulas  for  the 
quality  of  sensor  information,  which  is  equivalent  to  the  NCW  Con¬ 
ceptual  Framework’s  quality  of  organic  information  measure.  Of  the 
attributes  the  framework  defined  for  the  quality  of  organic  informa¬ 
tion,  we  provide  metrics  for  three:  completeness,  correctness,  and 
currency. 
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Figure  S.2 — The  Information  Superiority  Value  Chain 


Completeness.  We  examined  three  aspects  of  completeness:  the 
number  of  enemy  units  detected,  the  features  reported  for  the  units 
detected,  and  the  sensor  suite  coverage  area.  For  sensor  information 
to  be  complete,  all  features  of  all  units  in  the  relevant  ground  truth 
must  be  known,  and  the  entire  area  of  operations  must  be  under  sen¬ 
sor  observation.  The  suggested  completeness  metric  has  two  com¬ 
ponents,  both  of  which  are  between  0  and  1:  cx  is  the  fraction  of 
enemy  units  detected  (as  specified  in  F0),  and  c2  is  the  fraction  of  the 
area  of  operations  covered.  We  then  have  the  following  transfer 
function  that,  using  F0  as  an  input,  combines  these  two  components 
to  produce  a  0-1  completeness  metric:  Qcom(F,  |F0)  =  cfl-e~C2).  Here, 
F:  is  the  CROP  as  detected  by  the  sensors.1 

Correctness.  The  metrics  we  suggest  for  correctness  either  support 
controlled  experiments  or  support  actual  operations  (in  which  ana¬ 
lysts  can  only  approximate  ground  truth  from  sensor  inputs).  In 


Hhe  body  of  the  report  presents  rationales  for  all  the  metrics’  functional  forms. 
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either  case,  correctness  is  taken  to  mean  the  degree  to  which  the  true 
target  features  approximate  their  ground-truth  values.  Estimation 
theory  is  one  way  to  assess  the  deviation  from  ground  truth  for  con¬ 
trolled  experiments.  Since  an  unbiased  estimator  of  a  parameter  is 
one  whose  expected  value  matches  the  true  parameter,  the  differ¬ 
ence  between  the  estimate  and  the  known  ground  truth  appears  to 
be  a  suitable  metric  to  measure  correctness.  In  general,  if  A  is  a  mea¬ 
sure  of  nearness,  then  Qcor(F]|F0)  =  e~A  is  the  transfer  function  we 
used  to  map  A  to  a  0-1  metric. 

Assessing  correctness  in  support  of  operations  implies  that  ground 
truth  is  not  known.  In  this  case,  we  cluster  the  detections  geographi¬ 
cally  using  a  pattern-classification  technique  and  then  calculate  the 
variance  within  the  cluster.  For  a  location  estimate,  the  variance  is 
expressed  in  terms  of  a  covariance  matrix.  The  determinant  of  that 
matrix  is  a  measure  of  precision  and  therefore  a  measure  of  correct¬ 
ness.  The  determinant  is  p  =  S4,  where  S2  is  the  sample  variance  in 
both  the  x  and  y  directions.  Qcor(F|  |F0)  =  e~P  is  the  transfer  function 
we  used  to  produce  a  0-1  correctness  metric. 

Fusion  Metrics 

In  the  architecture  we  present  here,  the  sensors  transmit  their  read¬ 
ings  to  a  series  of  fusion  facilities,  each  of  which  focuses  on  a  single 
intelligence  discipline.  Each  facility  submits  its  fused  reports  to  a 
single  central  fusion  facility,  which  combines  the  sensor  inputs  into  a 
single,  common,  relevant  picture  of  the  battlespace:  the  fused  CROP. 
This  subsection  develops  metrics  for  the  quality  of  the  fused  CROP, 
which  is  equivalent  to  part  of  the  NCW  Conceptual  Framework’s 
quality  of  individual  information  measure.2  As  noted,  we  assumed 
that  the  underlying  network  transmits  the  sensor  readings  to  the 
fusion  facilities  perfectly. 

Fusion  includes  the  correlation  and  analysis  of  data  inputs  from 
supporting  sensors  and  sources.  Fusion  occurs  at  several  levels,  from 


2The  quality  of  individual  information  measure  is  a  multidimensional  array  measure, 
with  the  entries  along  one  dimension  corresponding  to  the  quality  of  information  seen 
by  each  individual.  Further,  one  of  those  “individuals”  is  a  user  at  the  central  fusion 
facility,  who  directly  sees  the  fusion  facility’s  output.  This  part  measures  the  Quality  of 
individual  information  as  perceived  by  that  user.  The  next  section  measures  the 
quality  of  individual  information  perceived  by  users  away  from  the  central  facility. 
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the  simple  combining  of  tracks  and  identity  estimates  to  assessments 
of  enemy  intent.  Our  focus  here  is  on  the  lower  levels  of  fusion, 
which  seek  to  improve  the  accuracy  and  completeness  of  the  sensor 
reports  on  enemy  units’  features. 

Completeness  in  the  fusion  subdomain  focuses  on  the  number  of 
sensor-detected  enemy  units  that  have  been  classified,  i.e.,  described 
in  terms  of  their  relevant  features.  The  number  of  enemy-unit  fea¬ 
tures  the  fusion  facilities  can  classify  depends  on  the  architecture  of 
the  fusion  suite,  the  degree  of  automation  used,  and  the  ability  of  the 
system  to  retask  the  sensors.  The  proposed  formula  for  a  0-1  com¬ 
pleteness  metric  is 


Qcom  (^2 


Fo-Fi)  = 


-nil  M) 


where  k  is  the  number  of  subsidiary  fusion  facilities,  c,  is  the  fraction 
of  the  detected  enemy  units  that  fusion  facility  i  can  classify  per  unit 
of  time,  cc  is  the  fraction  the  central  processing  facility  can  process, 
and  F2  is  the  CROP  after  it  has  been  through  the  fusion  process. 

Correctness  in  the  fusion  subdomain  measures  how  close  the  fused 
estimate  for  each  enemy  unit  feature  is  to  ground  truth.  That  is,  how 
accurate  are  the  classifications  of  the  reported  detections?  One  way 
we  might  address  this  problem  is  to  examine  the  variance  in  the  fea¬ 
ture  estimates  for  each  reported  unit.  This  results  in  the  following 
formula  for  a  0-1  correctness  metric: 


QcorA^0^)  =  lU^IPH°Jje~Si  ■ 

In  this  formulation,  w-,  and  coj  are  weights.  The  former  accounts  for 
the  relative  importance  of  the  reported  enemy  unit,  and  the  second 
accounts  for  the  relevant  importance  of  the  features  being  reported. 
The  values  of  Sj  are  sample  standard  deviations  for  each  of  the  p  fea¬ 
tures  for  a  given  enemy  unit,  derived  from  the  number  of  reports 
arriving  on  the  unit.  The  second  subscript  on  Q  is  used  to  distinguish 
this  correctness  transformation  from  the  tracking  metric  discussed 
next. 

An  additional  task  is  measuring  how  well  we  are  able  to  track  enemy 
units.  The  correctness  of  the  tracks  of  enemy  units  can  be  measured 
in  terms  of  the  number  of  previous  tracks  that  have  been  confirmed 
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on  the  present  scan  and  the  number  of  new  tracks  initiated.  The 
tracking  portion  of  the  correctness  component  of  the  transformation 
function  is  taken  to  be  Qcor 2(F2|F0,F1)  =  T,  where  T  is  the  fraction  of 
the  enemy  units  that  correlate  with  previous  tracks. 

Combining  the  two  correctness  metrics  using  an  importance  weight, 
0  <  co  <1  yields  the  following  for  the  correctness  component  of  the 
transformation  function: 

Qcor  (F2  |  F0  ,  Fj )  =  mW  +  (l  -  co)r , 


where  W=  Qcor,i(iyFo,Fi). 

Finally,  an  appropriate  0-1  metric  for  the  currency  attribute  of  qual¬ 
ity  of  the  fused  CROP  is  Qcur(T^|F0,Fi)  =  e~‘,  where  t  is  the  total  time 
required  to  update  the  fused  CROP.  This  function  emphasizes  the 
importance  of  updating  the  fused  CROP  quickly. 

Network  Metrics 

Following  fusion,  the  architecture  distributes  the  fused  CROP  to  the 
force  network’s  users,  resulting  in  the  observed  CROP.  Here,  we  pro¬ 
vide  metrics  for  the  quality  of  the  observed  CROP,  which  is  the 
remainder  of  this  report’s  instantiation  of  the  NCW  Conceptual 
Framework’s  quality  of  individual  information  measure.3  In  these 
calculations,  we  allow  the  network  to  incur  errors  and  delays  in  dis¬ 
tributing  the  CROP.  Thus,  although  we  do  not  specifically  incorpo¬ 
rate  the  NCW  Framework’s  Degree  of  Networking  and  Degree  of 
Information  “Shareability”  metrics  in  this  report,  these  metrics 
would  directly  influence  the  parameters  of  the  functions  used  to 
generate  the  quality  of  the  observed  CROP  metrics. 

Thus,  completeness  here  measures  how  well  the  communications 
network  accommodates  the  transmission  of  relevant  aspects  of  the 
CROP  to  each  user.  A  metric  for  this  measure  is  the  probability  that 
all  users  will  receive  the  CROP.  This  is  an  assessment  of  the  network’s 


3This  section  describes  how  to  calculate  the  quality  of  individual  information  metrics 
for  those  users  not  at  the  central  fusion  facility,  who  must  receive  the  CROP  over  the 
network. 
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reliability  in  terms  of  its  robustness.  The  resulting  completeness 
metric  has  the  following  formula: 

Qcom  (  ^3  I  F()  >  Pi  >  ^2  )  =  II?=1  Pi  • 

In  this  formulation,  k  is  the  number  of  users  of  the  CROP,  and  pt  rep¬ 
resents  the  probability  that  user  i  will  receive  the  CROP. 

Network  correctness  is  an  assessment  of  the  likelihood  that  CROP 
users  receive  the  distributed  information  without  degradation.  One 
way  to  measure  this  is  to  use  the  probability  of  correct  message 
receipt  (PCMR).  The  PCMR  is  a  conditional  probability  that  the  mes¬ 
sage  sent  will  be  the  message  received.  The  probability  that  user  i  will 
receive  the  CROP  (or  a  portion  of  it)  as  transmitted  is  P;(F3,F2)  = 
P(F2)P,:(F3|F2),  where  P,:(F3|F2)  =  p.  We  therefore  get  the  following 
PCMR  for  user  i: 


PCMR,-  =  T/(f3  , F2 )  =  p(f2  )p(- , 

where  P(F2)  is  the  probability  that  a  user  receives  the  CROP  without 
error,  given  that  the  user  receives  the  CROP.  Therefore,  our  formula 
for  a  0-1  metric  for  correctness  is 

Qcor(F3|F0,F1,F2)  =  niiPCMRi. 

The  end-to-end  time  required  to  transmit  the  CROP  from  the  central 
fusion  facility  to  the  users  serves  as  a  measure  of  network  currency. 
One  way  to  determine  this  is  to  calculate  the  average  of  all  paths 
from  the  source  to  the  user.  The  overall  average  network  transmis¬ 
sion  delay,  then,  is  taken  to  be  the  average  of  these  times,  t ,  so  that  a 
0-1  metric  for  currency  is 

Qcur[^3  I  Po’Pl’^)  =  e  ■ 


Shared  Information 

Shared  information  is  an  essential  ingredient  to  ensure  effective  col¬ 
laboration.  Recall  that  the  CROP  users  receive  is  the  observed  CROP. 
Matrix  F2  represents  the  fused  CROP.  Each  user’s  observed  CROP  is  a 
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subset  of  the  fused  CROP.  The  overlap  among  these  subsets  consti¬ 
tutes  the  information  shared  among  the  users.  Information  not  in  the 
overlap  has  the  potential  to  be  shared  through  the  process  of  collab¬ 
oration.  The  ability  to  collaborate  therefore  has  the  potential  to 
increase  the  amount  of  information  shared  among  the  users,  thus 
contributing  to  shared  situational  awareness. 

Since  “shared  information’’  applies  to  subsets  of  the  observed  CROP, 
the  quality  measures  for  Quality  of  Shared  Information  are  equiva¬ 
lent  to  those  for  the  Quality  of  the  Observed  CROP.  A  new  attribute, 
however,  is  the  extent  to  which  the  observed  CROP  is  shared.  The 
body  of  the  report  discusses  various  set-theoretic  metrics  for  deter¬ 
mining  the  extent  of  information  sharing. 

THE  COGNITIVE  DOMAIN 

In  the  information  domain,  the  data  collected  on  the  physical 
domain  are  processed  and  disseminated  to  friendly  users.  In  the 
cognitive  domain,  the  products  of  the  information  domain  are  used 
to  take  decisions.  The  mental  processes  that  transform  CROP  into  a 
decision  and  a  subsequent  action  depend  on  a  range  of  factors,  a  few 
of  which  are  psychological.  The  cognitive  processes  that  transform 
the  CROP  into  a  decision  and  subsequent  action  must  be  described 
for  participants  in  the  decision  process,  both  as  individuals  and  as 
interacting,  collaborating  members  of  a  decisionmaking  team.  In  this 
report,  we  restrict  our  attention  in  the  cognitive  domain  to  how  well 
users  can  assess  the  situation  presented  to  them  through  the 
observed  CROP.  With  respect  to  the  NCW  Conceptual  Framework, 
we  restrict  our  attention  to  the  Individual  and  Shared  Awareness 
measures,  which  are  subsets  of  the  framework’s  Individual  and 
Shared  Sense-Making  measures,  respectively. 

Modeling  Individual  Situational  Awareness 

Several  factors  influence  what  it  will  take  for  an  individual  decision¬ 
maker  to  correctly  assess  the  situation  presented  to  him.  Among 
these  is  the  quality  of  the  information  presented.  This  metric  assesses 
the  degree  to  which  the  decisionmaker  is  aware  of  the  situation  facing 
him,  emphasizes  the  use  of  the  individual  components  of  the  CROP, 
and  includes  a  reference  to  the  ability  of  the  individual  decision- 
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maker.  It  is  interpreted  to  be  the  fraction  of  the  observed  CROP  the 
decisionmaker  realizes. 

We  developed  an  agent  representation  of  a  decisionmaker  using 
combinations  of  capability  attributes  (education  and  training,  expe¬ 
rience)  and  defined  two  discrete  points  for  each  attribute.  From  this, 
we  produced  four  decision  agents  possessing  these  attributes  at  one 
of  the  two  points.  The  agents  suggest  a  functional  relationship  in 
which  the  dependent  variable  is  “degree  of  awareness”  and  the  inde¬ 
pendent  variables  are  information  quality  measures  (completeness 
in  this  case). 

The  end  result  of  this  process  is  an  explicit  relationship  between  the 
quality  of  the  observed  CROP  and  the  ability  of  the  decisionmaker. 

Modeling  Shared  Situational  Awareness 

To  describe  shared  situational  awareness,  we  augmented  the  indi¬ 
vidual  shared  awareness  model  by  representing  the  complex  inter¬ 
actions  in  situations  involving  more  than  one  individual.  The  metric 
we  chose  for  this  is  the  fraction  of  fused  feature  vectors  in  the  observed 
CROP  that  members  of  a  team  realize  similarly,  whether  or  not  they 
collaborate.  This  metric  emphasizes  the  importance  of  individual  sit¬ 
uational  awareness  and  allows  agreement  to  exist  even  when  indi¬ 
vidual  decisionmakers  have  not  collaborated. 

We  hypothesized,  however,  that  when  collaboration  is  used,  it  is  criti¬ 
cal  for  determining  shared  situational  awareness.  We  focused  on 
assessing  the  important  attributes  that  affect  teams  that  do  collabo¬ 
rate  and  therefore  have  either  positive  or  negative  effects  on  the 
degree  of  shared  awareness.4 

One  ingredient  of  the  shared  situational  awareness  process  is  the 
concept  of  a  common  ground.  For  our  purposes,  this  term  refers  to 
the  knowledge,  beliefs,  and  suppositions  that  team  members  believe 
they  share.  During  a  team  activity,  therefore,  common  ground  accu¬ 
mulates  among  team  members. 


4Note  that  these  attributes,  and  the  effectiveness  of  collaboration  in  general,  are  part 
of  the  NCW  Conceptual  Framework’s  Quality  of  Interactions  measure. 
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We  further  hypothesized  that,  to  be  effective,  collaboration  requires 
both  the  development  of  common  ground  among  collaborators  and 
familiarity  with  the  capabilities  of  other  collaborators.  Common 
ground  does  not  develop  instantaneously  when  there  is  collabora¬ 
tion;  there  is  a  period  of  “initial  calibration”  during  which  partici¬ 
pants  “tune  in”  to  each  other  and  move  from  a  state  of  common 
sense  to  states  of  common  opinion  and  common  knowledge. 

A  structural  model  for  defining  and  analyzing  this  phenomenon  is  a 
transactive  memory  system,  defined  as  a  set  of  individual  memory 
systems  in  combination  with  the  communication  that  takes  place 
between  individuals.  It  is  concerned  with  the  prediction  of  group 
(and  individual)  behavior  through  an  understanding  of  how  groups 
process  and  structure  information. 

Information  can  be  stored  and  retrieved  internally  by  an  individual 
according  to  the  individual’s  encoding,  storage,  and  retrieval  pro¬ 
cesses.  If  an  individual  stores  information  externally,  the  storage  and 
retrieval  process  must  also  include  the  location  of  the  information.  If 
externally  stored  information  resides  in  another  person,  a  transactive 
memory  system  exists.  Individuals  can  be  assigned  as  information 
stores  because  of  their  personal  expertise  or  through  circumstantial 
knowledge  responsibility.  Each  individual  participating  in  the  trans¬ 
active  memory  has  a  set  of  memory  components.  These  memory 
components  capture  the  key  elements  of  the  collaboration.  They  rep¬ 
resent  information  that  some  individuals  store  externally  in  other 
individuals  and  some  individuals  retain  on  behalf  of  other  individu¬ 
als  in  the  transactive  memory  system.  There  can  be  direct  links 
between  an  individual  and  the  retrieval  of  a  memory  item  and  there 
can  be  indirect  links  that  take  “hops”  through  the  transactive  mem¬ 
ory  system  until  the  memory  item  is  accessed. 

As  participants  develop  stronger  relationships  with  other  partici¬ 
pants  through  repeated  or  continued  team  interaction,  the  links 
between  the  participants  become  stronger.  This  suggests  a  second 
common  ground  hypothesis:  The  completeness  of  the  system  for 
recording  and  retrieving  information  depends  on  how  frequently  the 
team  has  recently  collaborated.  This  concept  is  referred  to  as  “team 
hardness.” 
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A  time-dependent  functional  model  for  team  hardness  is 
0  <  TM{T)  <  1,  where  TM(T)  is  a  function  whose  values  are  between  0 
and  1,  t  represents  the  time  elapsed  since  the  start  of  the  operation, 
and  x  represents  the  length  of  time  the  team  has  been  training  or 
operating  together,  and  T-x  +  t. 

Consensus  plays  a  central  role  in  developing  a  transactive  memory 
system.  It  is  the  majority  opinion  of  a  team  arrived  at  through  active 
collaboration.  Its  definition  implies  the  existence  of  shared  situational 
awareness.  Noting  that  not  all  collaborating  individuals  have  to  agree 
before  a  decision  and  subsequent  action  can  take  place,  we  are  inter¬ 
ested  in  a  measure  of  the  degree  of  consensus.  We  hypothesized  that 
the  degree  of  consensus  can  he  estimated  by  the  number  of  pairwise 
combinations  of  collaborating  individuals  who  interpret  feature  vec¬ 
tors  similarly. 

Models  of  shared  situational  awareness  integrate  the  modeling  pro¬ 
posed  earlier.  First,  we  placed  the  individual  in  a  team  and  measured 
his  situational  awareness  in  a  team  setting.  Note  that  this  is  not  the 
same  as  team  awareness  but  is  rather  the  effect  of  team  dynamics  on 
an  individual  member  of  a  collaborative  decisionmaking  process. 
The  contribution  is  essentially  derivative  of  the  transactional  mem¬ 
ory  function  and,  therefore,  team  hardness.  Second,  we  addressed 
the  consensus  that  develops  among  collaborating  individuals  and  its 
effects  on  the  team’s  shared  situational  awareness.  Finally,  we 
accounted  for  the  diversity  of  decision- agent  capabilities  among  the 
collaborators  that  results  in  our  composite  model  for  the  degree  of 
shared  situational  awareness. 

FUTURE  DIRECTIONS 

As  suggested,  this  work  is  clearly  incomplete.  We  have  described  a 
mathematical  framework  that  might  be  used  to  develop  detailed 
mathematical  quantities  that  represent  what  are  generally  consid¬ 
ered  qualitative  concepts.  In  some  cases,  data  may  exist  in  the  mili¬ 
tary  C4ISR  community  to  confirm  or  disconfirm  both  the  process  and 
any  of  our  examples.  In  these  cases,  locating  and  assessing  the  data 
are  required.  Where  data  do  not  exist,  further  experimentation  or 
historical  analysis  will  be  required. 
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Much  remains  to  be  done  in  the  cognitive  domain.  The  relationship 
between  information  quality  and  situational  awareness  is  the  first 
step  in  the  decisionmaking  process.  Further  work  is  needed  to  codify 
the  relationship  between  situational  awareness  and  the  ability  of  the 
decisionmaker  to  make  inferences  from  the  CROP — that  is,  his 
understanding  of  the  situation. 

Several  techniques  might  be  used  to  advance  our  knowledge  in  this 
important  area.  Among  these  are  the  following: 

•  Data  fitting.  We  can  use  existing  data  either  to  confirm  the  valid¬ 
ity  of  the  relationships  suggested  in  this  work  or  to  suggest  differ¬ 
ent  relationships. 

•  Experimentation.  Experiments  might  provide  additional  insights 
about  the  relationships  between  information  quality  and  aware¬ 
ness.  For  example,  we  could  select  decisionmakers  that  have 
various  combinations  of  awareness  characteristics.  The  degree  to 
which  they  are  able  to  realize  enemy  intent  from  what  is  pre¬ 
sented,  then,  is  an  indicator  of  their  level  of  awareness. 

•  Decision  and  action.  The  link  between  awareness  and  decision 
needs  to  be  established.  The  level  of  awareness  affects  the  ability 
to  understand,  i.e.,  to  draw  inferences  about  the  CROP,  such  as 
enemy  intent.  The  inferences,  in  turn,  affect  the  decision  to  be 
taken  and  therefore  the  subsequent  actions  ordered. 

•  Historical  analyses.  Analysis  of  past  battles  is  an  important 
source  of  insight  into  the  value  of  information.  Considerable  data 
is  available  from  various  sources  that  can  provide  insights  into 
the  relationship  between  the  quality  of  information  and  the  level 
of  awareness. 

•  Gaming.  It  is  also  possible  to  use  game  theory  to  illustrate  certain 
effects  of  information  imbalances  between  two  opponents.  In  a 
two-sided  game,  each  side  strives  to  obtain  high-quality  informa¬ 
tion.  At  the  same  time,  each  side  attempts  to  ensure  that  the 
opponent’s  information  is  of  low  quality.  Several  pairings  of 
players  with  varying  awareness  characteristics  might  then  be 
played  against  the  various  information- quality  levels.  In  this  way, 
a  link  is  established  from  information  quality  to  awareness  to 
decisions  and,  finally,  to  outcome. 
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Chapter  One 

INTRODUCTION 


It  is  the  quality  of  our  work  that  will  please  God  and  not  the 
quantity. 

— Mahatma  Gandhi 


The  military  is  formulating  new  visions,  strategies,  and  concepts  that 
capitalize  on  emerging  information- age  technologies  to  provide  its 
warfighters  with  significantly  improved  capabilities  to  meet  the 
national  security  challenges  of  the  21st  century.  New,  networked 
command,  control,  communications,  computers,  intelligence, 
surveillance,  and  reconnaissance  (C4ISR)  capabilities  promise  infor¬ 
mation  superiority  and  decision  dominance  that  will  enhance  the 
quality  and  speed  of  command  and  enable  revolutionary  warfighting 
concepts.  Assessing  the  contribution  of  C4ISR  toward  achieving  a 
network-centric  warfare  (NCW)  capability  is  a  major  challenge  for 
the  Department  of  Defense  (DoD)  because  of  the  multiplicity  of 
interacting  factors  and  the  lack  of  understanding  of  the  fundamen¬ 
tals  associated  with  information-superiority  concepts.  DoD  has 
embarked  on  a  journey  of  exploration  to  discover  how  to  create  and 
leverage  information  superiority  by  characterizing  the  conditions 
under  which  it  can  be  achieved  and  under  which  a  competitive 
advantage  can  be  gained.  Much  as  in  the  development  of  a  new 
branch  of  science,  this  requires  defining  concepts,  metrics,  hypothe¬ 
ses,  and  analytical  methodologies  that  can  be  used  to  focus  research 
efforts,  identify  and  compare  alternatives,  and  measure  progress. 
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In  response  to  this  need,  DoD’s  Office  of  Force  Transformation,  in 
conjunction  with  the  Office  of  the  Assistant  Secretary  of  Defense  for 
Networks  and  Information  Integration  (ASD  Nil),  has  been  evolving  a 
conceptual  NCW  framework  for  assessment  that  includes  measures, 
general  forms  for  metrics,  and  relationships  between  the  measures 
and  metrics.  The  NCW  Conceptual  Framework  is  intended  for 
exploratory  analysis  of  potential  C4ISR  architectures  and  for  guiding 
C4ISR  experiments  in  terms  of  identifying  the  measures  for  which  to 
collect  experimental  data.1 

The  technical,  quantitative  application  of  this  framework  requires 
developing  mathematical  models  that  reflect  the  performance  of  the 
enabling  C4ISR  processes,  architectures,  and  capabilities,  as  well  as 
developing  a  hypothesis  identifying  the  key  variables  to  measure, 
and  their  effects.  Consequently,  this  report  describes  a  methodology 
for  developing  and  linking  relevant  mathematical  models.  It  provides 
mathematical  models  corresponding  to  specific  C4ISR  architectures 
and  illuminates  the  data  needed  to  apply  the  NCW  Conceptual 
Framework  in  a  specific  context. 

RESEARCH  OBJECTIVES 

An  important  first  step  is  to  develop  a  better  understanding  of  how 
improved  C4ISR  capabilities  and  related  changes  in  command  and 
control  processes  contribute  to  the  achievement  of  such  core  NCW 
concepts  as  situational  awareness,  shared  situational  awareness,  and 
synchronization.  The  next  step  is  to  assess  how  well  the  core  con¬ 
cepts  affect  command  and  control  operational  concepts  and,  in  turn, 
the  success  of  military  operations.  Establishing  a  quantifiable  link 
between  improved  C4ISR  capabilities  and  combat  outcomes  has 
been  extremely  elusive  and  is  therefore  a  major  challenge. 

The  primary  objective  of  our  work  was  to  develop  a  clearly  articu¬ 
lated  mathematical  framework  that  would  explore  how  certain  major 
factors  would  affect  the  hypotheses  related  to  NCW.  The  scope  of  the 
research  is  restricted  to  the  NCW  Conceptual  Framework’s  key  mea¬ 
sures  related  to  information  superiority,  which  Joint  Publication  1-02 


1The  most  recent  public  document  on  the  NCW  Conceptual  Framework  is  Signori  et 
al.  (2002). 


Introduction  3 


defines  as  “[t]hat  degree  of  dominance  in  the  information  domain 
which  permits  the  conduct  of  operations  without  effective 
opposition”  (DoD,  2003,  p.  255). 

In  particular,  we  examined  how  the  quality  of  information 
(individual  and  shared)  a  C4ISR  network  provides  affects  the  quality 
of  situational  awareness  (also  individual  and  shared).  In  the  NCW 
Conceptual  Framework,  the  Quality  of  Information  and  Quality  of 
Awareness  are  a  vector  of  submeasures.  The  research  focuses  on  a 
subset  of  these  submeasures,  notably  completeness,  correctness,  and 
currency. 

The  modeling  approach  was  to  study  a  chain  of  information  quality 
and  awareness  quality  metrics  in  three  domains:  physical,  informa¬ 
tion,  and  cognitive.  The  models  assumed  that  a  specific  type  of  C4ISR 
architecture  is  being  employed,  in  which  an  array  of  sensors  transmit 
data  about  the  battlespace  to  a  central  fusion  facility  that  distributes 
a  resulting  common  relevant  operating  picture  (CROP)  to  force 
members.  The  force  members  then  mentally  interpret  the  CROP  they 
receive  (creating  awareness)  and  collaborate  with  other  force 
members  to  improve  everyone’s  awareness. 

This  report  emphasizes  the  development  of  the  mathematical 
framework.  Although  we  include  several  measures  and  mathemati¬ 
cally  defined  metrics,  they  should  not  be  considered  the  results  of 
settled  research.  They  are  necessarily  simple  and  illustrative  rather 
than  general  and  widely  applicable. 

The  quantitative  methodology  and  illustrative  mathematical  repre¬ 
sentations  here  relate  to  force-on-force  combat  operations  as 
opposed  to  a  broader  spectrum  of  operation  that  includes  aid  to  civil 
authorities,  humanitarian  relief,  and  peacekeeping  operations.  How¬ 
ever,  aspects  of  the  approach  provide  a  basis  for  informed  dialog 
that,  in  our  view,  will  eventually  lead  to  more-comprehensive  and 
better-validated  capabilities  to  quantitatively  explore  the  influence  of 
improved  C4ISR  systems  and  processes  on  operational  outcome. 

ANALYTICAL  FRAMEWORK 

In  addition  to  the  NCW  Conceptual  Framework,  the  research 
reported  here  builds  on  the  work  of  the  ASD  Nil  Information  Superi- 
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ority  Metrics  Working  Group  (ISMWG).  This  body  has  developed 
working  definitions,  specific  characteristics  and  attributes  of  key 
concepts,  and  the  relationships  among  them  that  are  needed  to  mea¬ 
sure  the  degree  to  which  information  superiority  concepts  are  real¬ 
ized  and  their  impact  on  the  conduct  and  effectiveness  of  military 
operations.  Such  an  endeavor  requires  a  common  language  and  a  set 
of  integrated  hypotheses,  as  well  as  the  metrics,  instruments,  and 
tools  to  collect  and  analyze  data,  such  as  those  suggested  here. 

We  begin  by  defining  a  reference  model  for  discussing  such  issues  in 
terms  of  three  domains:  that  of  ground  truth  (the  physical  domain); 
that  of  sensed  information  (the  information  domain);  and  that  in 
which  individual  situational  awareness,  shared  situational  aware¬ 
ness,  collaboration,  and  decisionmaking  occur  (the  cognitive 
domain).  The  C4ISR  process  is  seen  as  extracting  data  from  ground 
truth  and  processing  the  data  in  the  information  domain  to  produce 
a  CROP.  The  quality  of  the  CROP  combines  with  the  quality  of  team 
collaboration  to  heighten  (or  degrade)  shared  situational  awareness 
in  the  cognitive  domain.  The  ground  truth  data,  obtained  from  the 
collection  process  in  the  information  domain,  is  transformed  into  a 
CROP  that  contributes  to  situational  awareness  in  the  cognitive 
domain.  The  transformations  are  processes  that  include  data  collec¬ 
tion  and  processing,  data  fusion,  and  information  dissemination. 
These  processes  are  not  discussed  in  this  report.  We  assume  that 
they  are  performed  and  focus  instead  on  the  quality  of  the  informa¬ 
tion  and  products  they  generate.2 

LIMITATIONS 

This  report  focuses  on  the  collection  of  data;  the  processing  of  col¬ 
lected  data  to  produce  the  CROP;  the  dissemination  of  the  CROP 
from  the  various  fusion  facilities  to  the  ultimate  users;  the  experience 
of  the  decisionmaking  team;  the  quality  of  its  collaboration;  and, 
finally,  how  all  this  affects  shared  situational  awareness.  In  develop¬ 
ing  mathematical  representations  of  information  quality  and  the 
effects  of  collaboration,  we  focused  exclusively  on  battlefield  enti¬ 
ties — individual  units  and  weapon  systems.  The  possible  inferences 


2For  a  complete  discussion  of  C4ISR  information  processing  algorithms,  see  Perry  and 
Sullivan  (1999). 
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to  be  drawn  from  the  patterns  these  entities  exhibit,  although  very 
important,  are  beyond  the  scope  of  this  initial  work. 

We  also  do  not  treat  several  important  aspects  of  the  decisionmaking 
cycle,  such  as  decisionmaking  and  synchronization.  The  relationship 
between  awareness  and  understanding  and  how  understanding 
affects  decision  and  action  are  also  not  addressed.  Nevertheless,  this 
document  presents  a  new  and  important  methodology  for  assessing 
the  quality  of  information  and  intelligence,  surveillance,  and  recon¬ 
naissance  processes  in  general,  as  well  as  some  of  the  more  psycho¬ 
logical  aspects  of  the  decisionmaking  process.  It  is  based  on  sound 
mathematical  concepts  and  hypotheses  from  the  literature  and 
therefore  provides  a  foundation  for  further  inquiry.  The  illustrative 
mathematical  relationships,  however,  will  require  verification  and 
refinement  through  experimental  and  operational  data. 

Finally,  with  respect  to  the  NCW  Conceptual  Framework,  this  report 
is  restricted  to  the  information  and  awareness  measures.  The  report 
does  not  attempt  to  show  models  that  traverse  all  the  measures  in 
the  framework. 

ORGANIZATION  OF  THIS  REPORT 

Chapter  Two  outlines  the  analytic  framework  we  adopted  to  assess 
the  effects  of  information  quality  and  team  collaboration  on  shared 
situational  awareness  and,  eventually,  decision  and  execution.  We 
begin  by  describing  the  C4ISR  Information  Superiority  Reference 
Model  as  the  underlying  construct  that  describes  the  C4ISR  process. 
Chapter  Three  focuses  on  a  mathematical  framework  for  assessing 
the  contributions  of  both  the  physical  and  information  domains  to 
information  quality.  Chapter  Four  turns  to  the  more-psychological 
aspects  of  decisionmaking  and  assesses  the  contributions  of  the 
cognitive  domain  to  shared  situational  awareness.  Finally,  Chapter 
Five  addresses  future  work.  Three  appendices  are  included: 
Appendix  A  lists  the  definitions  of  important  terms;  Appendix  B 
records  a  few  example  mathematical  representations  for  the  metrics 
developed  in  the  main  text;  and  Appendix  C  describes  a  spreadsheet 
model  used  to  illustrate  the  framework  using  the  example  metrics 
listed  in  Appendix  B. 


Chapter  Two 

THE  ANALYTIC  FRAMEWORK 


A  decision  is  an  action  an  executive  must  take  when  he  has 
information  so  incomplete  that  the  answer  does  not  sug¬ 
gest  itself. 

— Arthur  William  Radford 


This  chapter  outlines  the  analytic  framework  we  adopted  to  assess 
the  effects  of  information  quality  and  team  collaboration  on  shared 
situational  awareness  and,  eventually,  decision  and  execution.  We 
begin  by  describing  the  underlying  construct  that  describes  the 
C4ISR  process  we  model  here:  the  C4ISR  Information  Superiority 
Reference  Model.  The  quality  of  the  information  flowing  through 
what  we  call  “the  information  value  chain”  is  transformed  to  produce 
an  overall  assessment  of  the  quality  of  the  CROP  generated.  This,  in 
turn,  affects  individual  situational  awareness  and,  subsequently, 
shared  situational  awareness  through  the  process  of  team  collabora¬ 
tion. 

THE  C4ISR  INFORMATION  SUPERIORITY  REFERENCE 
MODEL 

The  C4ISR  Information  Superiority  Reference  Model  (Figure  2.1)  is  a 
representation  and  extension  of  the  portion  of  the  NCW  Conceptual 
Framework  dealing  with  information  superiority  issues  (in  particular, 
the  information  and  awareness  measures).  It  represents  the  activities 
associated  with  collecting  data;  processing  information;  discerning 
enemy  intent,  plans,  and  physical  activities  (or  lack  thereof);  select- 
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ing  a  course  of  action;  and  monitoring  its  execution.  The  model  con¬ 
sists  of  three  “domains”  that  extend  from  the  battlefield  environment 
to  cognitive  awareness  of  the  battlefield  situation  and  decision.  The 
measures  and  metrics  proposed  here,  although  not  all-inclusive,  are 
presented  within  this  context. 

Both  sides  in  a  conflict  generally  have  different  perceptions  of  a  sin¬ 
gle  reality,  referred  to  as  the  situation.  Figure  2.1  depicts  how  the 
three  domains  contribute  to  this  perception.  The  major  activities 
performed  in  each  of  the  domains  are  listed  in  its  box.  The  physical 
domain  is  where  reality  or  ground  truth  resides. 

In  addition  to  physical  objects  ( entities ) — weapon  systems,  terrain 
features,  sensors,  etc. — the  physical  domain  also  contains  intangi¬ 
bles,  such  as  enemy  intent,  plans,  and  current  and  projected  activi¬ 
ties.  A  complete  assessment  of  the  situation  will  contain  estimates 
about  each.  As  mentioned  earlier,  however,  we  will  focus  on  entities 
only,  thus  reserving  the  others  for  future  research. 

RAND  MR1467-2.1 


Cognitive  domain 


NOTE:  The  activities  depicted  in  each  of  the  domain  “boxes”  may  not  be  complete. 
We  focus  on  those  activities  pertinent  to  our  research. 


Figure  2.1 — The  Information  Superiority  Reference  Model 
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In  the  information  domain,  data  are  extracted  from  the  physical 
domain  and  processed  to  form  structured  information  in  the  form  of 
a  CROP.  Three  primary  functions  are  performed  in  the  information 
domain:  collecting  data  through  the  use  of  sensors  and  sources, 
including  tasking  sensors  to  close  gaps  in  the  data;  processing  the 
data  through  the  fusion  process  to  produce  the  CROP;  and  dissemi¬ 
nating  relevant  parts  of  the  CROP  to  friendly  units.  The  last  step 
contributes  to  the  collaboration  process  in  the  cognitive  domain,  in 
which  the  shared  CROP  is  transformed  into  a  shared  awareness  of 
the  current  and  future  situations  that  can  be  used  to  gain  under¬ 
standing  of  threats  and  opportunities,  as  well  as  the  subsequent 
decisionmaking  regarding  an  appropriate  course  of  action. 

Finally,  the  human  activities  associated  with  using  the  information 
available  to  form  an  estimate  of  the  situation  take  place  in  the  cogni¬ 
tive  domain.  To  the  extent  that  decisionmaking  teams  exist,  they 
collaborate  to  form  a  level  of  situational  awareness.  In  addition  to  the 
CROP  produced  in  the  information  domain,  individual  team  mem¬ 
bers  and  the  decisionmaker  may  have  prior  information  from  such 
processes  as  the  information  preparation  of  the  battlefield  (IPB) 
available  to  support  their  deliberations.  Finally,  the  decisionmaker  is 
likely  to  have  concerns  and  expectations  about  the  performance  of 
his  own  forces,  as  well  as  those  of  the  enemy,  that  would  color  his 
assessment  of  the  situation  and  therefore  his  decision.  These  are 
depicted  as  emanating  directly  from  the  physical  domain. 

THE  C4ISR  ARCHITECTURE 

Employing  the  NCW  Conceptual  Framework  and  the  Information 
Superiority  Reference  Model  requires  a  C4ISR  architecture.  The 
architecture  used  in  this  report  may  be  thought  of  as  a  linear  process 
with  six  steps: 

0.  Ground  Truth.  The  architecture  begins  with  acceptance  of  the 
existence  of  physical  ground  truth.  As  the  previous  chapter  noted, 
we  restricted  our  consideration  of  ground  truth  to  battlefield  enti¬ 
ties,  such  as  individual  units  and  weapon  systems,  and  their 
physical  attributes. 

1.  Sensing.  The  architecture  first  uses  a  set  of  sensors  to  detect  the 
battlespace  entities  and  their  sensors. 
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2.  Fusion.  The  sensors  then  transmit  their  data  to  fusion  facilities. 
The  fusion  structure  used  in  this  report  is  centralized  and  has  two 
stages.  In  the  first  stage,  incoming  sensor  data  are  sent  to  one  of  lc 
fusion  facilities,  each  corresponding  to  a  different  intelligence 
discipline.  In  the  second  stage,  the  partially  fused  data  from  each 
discipline  are  forwarded  to  a  central  processing  facility,  which 
generates  a  single  CROP.  For  the  sake  of  simplicity,  we  assumed 
that  the  network  connections  transmitting  data  from  the  sensors 
to  the  fusion  facilities  are  flawless. 

3.  Distribution.  The  central  processing  facility  then  transmits  vari¬ 
ous  versions  of  the  CROP  to  the  network’s  users.  At  this  point,  we 
allowed  the  modeling  of  loss  of  service,  errors,  and  delays  in  the 
network. 

4.  Individual  Assessment.  Each  user  then  attempts  to  interpret  the 
CROP  he  has  received  to  achieve  some  level  of  realization  of  the 
battlespace. 

5.  Group  Assessment.  The  users  then  collaborate  with  each  other  in 
an  attempt  to  improve  their  realization  of  ground  truth  in  the  bat¬ 
tlespace.  This  report  models  the  effectiveness  of  collaboration  as  a 
function  of  the  skills  of  the  users  and  the  collaborative  group  as  a 
whole  but  does  not  examine  the  affects  of  the  network’s  commu¬ 
nications  tools  on  collaboration. 

THE  NCW  VALUE  CHAIN 

In  conjunction  with  the  NCW  Conceptual  Framework  and  the 
Information  Superiority  Reference  Model,  a  hierarchy  of  hypotheses 
regarding  NCW  concepts  is  emerging  as  the  focus  of  experimentation 
and  analysis.  Central  among  these  is  the  hypothesis  that  improved 
networked  C4ISR  capability  will  improve  information  quality  and 
shared  situational  awareness,  allow  more-dynamic  decisionmaking 
and  agile  force  synchronization,  and  will  ultimately  increase  force 
effectiveness  and  the  likelihood  of  a  successful  operational  outcome. 
The  basis  of  the  hypothesis  is  that  improved  networked  C4ISR  capa¬ 
bility  will  add  value. 

Consequently,  we  used  the  reference  model  to  establish  the  infor¬ 
mation  superiority  value  chain  and  examined  it  quantitatively  to 
determine  which  factors  have  the  greatest  payoff  and  the  conditions 
under  which  benefits  accrue.  Figure  2.2  depicts  the  initial  portion  of 
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this  value  chain,  which  represents  the  hypotheses  from  the  NCW 
Conceptual  Framework  that  are  associated  with  the  Information 
Superiority  Reference  Model  and  also  applies  directly  to  the  C4ISR 
architecture  discussed  above. 

In  particular,  the  value  chain  represents  the  processes  associated 
with  the  collection,  fusion,  and  dissemination  of  information,  as  well 
as  the  degree  to  which  the  information  contributes  to  shared  situa¬ 
tional  awareness  and,  ultimately,  to  the  decision  rendered,  focusing 
on  the  value  these  processes  add  (or  subtract)  at  each  step. 
Examination  of  the  value  chain  will  require  measuring  key  aspects  of 
the  end-to-end  process — not  only  the  quality  of  information  and 
degree  of  shared  situational  awareness  but  also  the  functional  per¬ 
formance  of  the  people  and  systems  that  transform  these  attributes 
along  the  way.  The  effects  of  such  functions  will  vary  with  the  degree 
of  integration  in  the  sensor  suite;  the  level  of  fusion  available  at  the 
fusion  centers;  the  connectivity  of  the  communications  network;  the 
capability  of  the  decisionmakers;  and  the  type  of  command  and  con¬ 
trol  process,  including  the  degree  of  collaboration. 
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Figure  2.2 — The  Information  Superiority  Value  Chain 


12  Exploring  Information  Superiority 


The  five  graphs  in  Figure  2.2  depict  illustrative  cascading  transfer 
functions  that  successively  map  performance  in  one  part  of  the  value 
chain,  as  well  as  exogenous  inputs,  to  performance  in  the  next  part  of 
the  value  chain.  The  emphasis  is  on  the  quality  of  the  processes,  not 
on  the  processes  themselves — on  “how  well  the  sensor  suite  per¬ 
forms”  and  not  on  “what  it  performs.”  The  first  transfer  function 
addresses  the  effect  of  sensor  performance  on  the  quality  of  the 
information  produced.  The  degree  and  nature  of  the  multisensor 
integration  can  be  altered  parametrically  to  obtain  the  shifts  in 
Quality  of  Sensor  Information  depicted  in  the  figure.  The  quality  of 
the  information  obtained  from  the  sensor  suite,  as  well  as  the  level  of 
fusion  (an  exogenous  input) ,  contributes  to  the  Quality  of  the  fused 
CROP.  In  the  figure,  the  levels  of  fusion  refer  to  the  extent  to  which 
the  fusion  facility  is  able  to  produce  not  only  information  on  battle¬ 
field  entities  but  also  assessments  of  enemy  intent.  The  observed 
CROP  depicted  in  the  next  chart  in  the  figure  refers  to  the  CROP  dis¬ 
tributed  to  the  ultimate  users,  generally  members  of  a  collaborative 
team.  The  Quality  of  the  Observed  CROP,  therefore,  depends  on  the 
communications  network  that  facilitates  its  transmission. 

The  versions  of  the  CROP  distributed  to  various  users  may  vary. 
Thus,  we  are  interested  in  the  similarities  between  the  users’ 
observed  CROPs.  Although  not  directly  shown  on  the  figure,  the 
Quality  of  the  Shared  CROP  measure  describes  the  consistency 
between  users’  observed  CROPs. 

In  the  next  chart  in  the  figure,  we  leave  the  information  domain  for 
the  cognitive  domain  by  depicting  the  relationship  between  the 
observed  CROP  and  individual  awareness.  Note  that  situational 
awareness  replaces  quality  of  information  as  the  dependent  variable. 
This  is  the  beginning  of  the  team  collaboration  process,  and  its 
effectiveness  will  vary  with  the  ability  of  the  individual  team  mem¬ 
ber.  Finally,  the  degree  to  which  the  individual  team  members  are 
aware  of  the  situation  affects  shared  situational  awareness.  The 
quality  of  the  collaboration  depends  on  several  factors,  such  as  the 
experience  of  the  team,  how  long  its  members  have  worked  together, 
and  the  personalities  and  position  of  its  members. 

The  value  chain  described  above  represents  an  application  of  part  of 
the  NCW  Conceptual  Framework.  The  differences  and  similarities 
are  as  follows: 
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•  Quality  of  Sensor  Information  implements  the  framework’s 
Quality  of  Organic  Information  measure  but  is  restricted  to  sen¬ 
sor  data. 

•  Quality  of  the  Fused  CROP  partially  implements  the  framework’s 
Quality  of  Individual  Information  measure  and  assumes  the  exis¬ 
tence  of  a  centralized  facility  responsible  for  fusing  the  sensed 
data  (see  below);  this  measure  captures  the  quality  of  the  infor¬ 
mation  a  user  at  this  central  facility  would  see. 

•  Quality  of  the  Observed  CROP  implements  the  rest  of  the 
framework’s  Quality  of  Individual  Information  measure.  The 
transfer  function  for  this  measure  incorporates  the  performance 
of  a  network  in  transmitting  the  CROP  to  various  users.  In  the 
framework,  network  performance  in  transmitting  particular 
pieces  of  information  comes  under  the  Degree  of  Information 
“Shareability”  measure.  Thus,  the  transfer  function  for  Quality  of 
the  Observed  CROP  models  the  dependency  in  the  framework 
between  the  shareability  and  quality  of  information. 

•  Quality  of  the  Shared  CROP  implements  the  framework’s 
Quality  of  Shared  Information  measure. 

•  Quality  of  Situational  Awareness  implements  the  Awareness 
submeasure  of  the  framework’s  Quality  of  Individual  Awareness 
measure;  this  report  does  not  cover  the  second  submeasure, 
Understanding. 

A  systematic  analysis  of  the  value  chain  helped  us  to  understand  the 
complexities  associated  with  the  end-to-end  C4ISR  process.  A  quanti¬ 
tative  methodology  that  would  account  for  links,  key  models,  and 
parameters  and  that  would  permit  examination  of  the  highly  nonlin¬ 
ear  effects  within  the  value  chain  would  facilitate  analysis.  As  an  ini¬ 
tial  step,  we  proposed  such  a  methodology  for  exploring  the  initial 
portion  of  the  value  chain  as  depicted  in  Figure  2.2,  which  we  discuss 
below. 

INFORMATION  QUALITY 

The  NCW  Conceptual  Framework  uses  eight  attributes  to  describe 
the  quality  of  information,  whether  the  information  is  organic,  indi¬ 
vidual,  or  shared.  The  attributes  are  divided  into  two  groups  of  four 
(see  Table  2.1): 
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Table  2.1 

Objective  and  Fitness  Measures 


Measure 

Definition 

Objective  measures 

Correctness 

The  extent  to  which  information  is  con¬ 
sistent  with  ground  truth 

Consistency 

The  extent  to  which  information  is  in 
agreement  with  related  or  prior  infor¬ 
mation 

Currency 

The  age  of  the  information 

Precision 

The  level  of  measurement  detail  in  an 
information  item 

Fitness  measures 

Completeness 

The  extent  to  which  information  relevant 
to  ground  truth  is  collected  ("relevant 
to  ground  truth”  depends  on  the  sce¬ 
nario) 

Accuracy 

The  appropriateness  of  the  precision  of 
information  to  a  particular  use 

Relevance 

The  proportion  of  information  collected 
that  is  related  to  the  task  at  hand 

Timeliness 

The  extent  to  which  the  currency  of 
information  is  suitable  to  its  use 

•  Objective  measures  can  be  used  regardless  of  the  particular  sce¬ 
nario 

•  Fitness-for-use  measures  require  a  particular  scenario  to  be 
defined.  Put  another  way,  the  objective  measures  are  context- 
free,  while  the  fitness  for  use  measures  are  context  dependent. 

The  three  measures  of  information  quality  we  selected  from  those 
listed  in  Table  2.1  were  completeness,  correctness,  and  currency.  We 
also  modified  the  definitions  slightly  for  our  use,  as  shown  in  Table 
2.2.  As  noted,  completeness  is  a  fitness-for-use  measure  because  its 
use  requires  determining  what  the  “relevant  ground  truth”  com¬ 
prises.  The  other  two  measures  are  objective,  although  one  can 
incorporate  contextual  information  in  defining  metrics  for  them.  For 
example,  if,  in  a  given  situation,  200-meter  accuracy  is  all  that  is 
needed,  that  can  be  made  to  equate  to  absolute  correctness. 
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Table  2.2 

Measures  of  Information  Quality 

Measure 

Description 

Completeness 

The  degree  to  which  the  information  is  free  of  gaps 
with  respect  to  the  relevant  ground  truth 

Correctness 

The  degree  to  which  the  information  agrees  with 
ground  truth 

Currency 

The  time  required  to  produce  a  CROP 

Completeness 

The  definition  of  completeness  in  Table  2.2  is  admittedly  restrictive. 
This  is  because  we  confined  our  definition  of  relevant  to  a  CROP  that 
consists  of  information  about  enemy  units  in  the  area  of  operations 
(AO) .  In  this  context,  having  information  about  all  relevant  features 
of  all  enemy  units  (no  gaps)  means  that  the  information  is  complete. 
In  many  scenarios,  we  can  further  restrict  relevance  to  certain  key 
features  about  the  enemy  units.  Here,  relevant  implies  that  com¬ 
pleteness  depends  on  the  situation,  the  command  level,  and  the 
scale  of  the  operation.  For  example,  if  we  are  interested  in  determin¬ 
ing  whether  an  enemy  army  is  on  the  move,  we  may  have  complete 
information  if  we  observe  only  a  few  vehicles  moving.  In  this  case, 
the  pattern  established  (relevant  entities  on  the  move)  is  enough  to 
assess  the  information  as  complete. 

Correctness 

Correctness,  as  defined  in  Table  2.2,  compares  what  is  observed,  pro¬ 
cessed,  and  disseminated  with  the  ground  truth.  This  applies  equally 
if  the  enemy  is  engaged  in  deception  or  not.  As  with  completeness, 
correctness,  in  this  context,  focuses  on  the  degree  to  which  what  is 
reported  is  close  to  truth.  For  example,  if  a  unit  is  on  the  move  and  if 
the  sensors  and  processing  facilities  produce  a  CROP  that  accurately 
records  the  location,  speed,  and  direction  of  travel  of  the  unit,  the 
information  is  taken  to  be  correct.  However,  we  recognize  that  this 
says  little  about  the  enemy  unit  commander’s  intentions.  Is  this  part 
of  a  feint?  Is  this  a  movement  to  contact  or  is  it  an  administrative 
march?  Answers  to  these  questions  are  arrived  at  by  examining  pat- 
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terns  and  drawing  inferences  from  them,  something  we  recognize 
requires  further  study. 

Currency 

Our  use  of  the  currency  measure  rather  than  timeliness  indicates 
that  we  are  only  interested  in  how  long  it  will  take  to  get  information 
to  the  users  not  whether  it  will  arrive  in  time.  Although  the  latter 
requires  that  we  calculate  the  former,  they  are  two  distinct  concepts. 
However,  like  the  other  measures,  currency  can  be  viewed  in  context. 
If  we  assume  that  faster  is  better,  information  that  can  be  processed 
quickly  has  high  quality.  However,  asking  how  fast  is  fast  enough 
introduces  context,  which  means  we  are  really  dealing  with  timeli¬ 
ness. 

A  QUANTITATIVE  METHODOLOGY 

A  mathematical  representation  of  the  transformations  depicted  in 
Figure  2.2  must  account  for  the  changes  in  information  quality  and 
situational  awareness  associated  with  the  processes  depicted.  In  sub¬ 
sequent  chapters  and  in  Appendix  B,  we  suggest  some  simple  con¬ 
ceptual  models  for  assessing  information  quality  and  situational 
awareness.  The  models  incorporate  metrics  that  measure  the  quality 
of  products  that  the  processes  depicted  in  Figure  2.2  generate,  as  well 
as  the  effects  of  collaborative  processes  on  situational  awareness. 
These  models  can  be  linked  using  a  basic  quantitative  methodology. 
The  intent  is  to  illustrate  the  methodology  and  provide  a  modeling 
baseline  for  further  extension  as  data  become  available  and  as  theory 
is  fully  incorporated,  particularly  where  less  tractable  parameters  are 
involved.  To  the  extent  that  data  are  available  or  that  more-detailed 
models  exist,  the  developed  models  can  be  refined  and  tailored  to 
the  problem  at  hand.  However,  depending  on  the  particular  prob¬ 
lem,  other  models  will  likely  be  more  appropriate.  Exploratory 
modeling  and  analysis  provide  tools  for  examining  the  effects  of  a 
variety  of  alternative  models  to  account  for  a  range  of  behavior  or 
uncertainty  associated  with  complex  C4ISR  systems  (see  Bankes, 
1993). 

The  methodology  has  three  segments.  The  first  quantifies  key  fea¬ 
tures  of  the  real  world;  to  begin  with,  these  are  confined  to  battlefield 
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entities,  such  as  units  and  weapon  systems.  As  mentioned  earlier,  we 
have  deferred  the  assessment  of  patterns  that  reveal  enemy  intent  to 
future  research.  The  second  segment  quantifies  the  quality  of  infor¬ 
mation  as  it  transits  the  sensors,  fusion  centers,  and  distribution 
networks  of  the  C4ISR  infrastructure.  The  third  quantifies  the  influ¬ 
ence  of  the  resulting  quality  of  information  on  the  degree  of  shared 
situational  awareness  through  the  collaboration  process.  In  each 
case,  we  defined  metrics  that  quantitatively  characterize  the  key  fea¬ 
tures  associated  with  the  situation,  as  well  as  the  performance  of  the 
people  and  systems  that  process  the  information.  In  the  case  of  the 
degree  of  shared  situational  awareness,  we  postulated  that  the 
transfer  functions  that  reflect  the  effects  of  the  quality  of  information 
would  vary  with  a  few  dominant  parameters  associated  with  individ¬ 
ual  and  group  behavior.  We  chose  these  parameters  on  the  basis  of  a 
brief  review  of  the  relevant  literature. 

For  the  quality  of  information  the  infrastructure  produces,  we  devel¬ 
oped  operationally  meaningful  metrics  so  that  they  or  their  proper¬ 
ties  could  be  calculated  using  estimation  theory.  We  used  a  different 
approach  for  assessing  situational  awareness.  The  experience  of  the 
members  of  the  collaborative  team  and  their  effectiveness  working  as 
a  team  were  the  primary  measures.  In  any  case,  however,  the  metrics 
were  reduced  to  a  number  between  0  and  1.  It  seemed  intuitive, 
therefore,  to  view  the  value  chain  as  a  series  of  conditional  transfer 
processes,  i.e.,  the  quality  of  the  information  the  fusion  process  pro¬ 
duces  is  conditioned  on  the  quality  of  the  information  the  sensor 
suite  produces.  This  allowed  us  to  construct  a  series  of  transfer  func¬ 
tions  that  compute  numerical  metrics  by  applying  a  conditional 
product-form  model. 

The  system  produces  information  about  the  enemy  units  and 
weapon  systems  (battlefield  entities) .  Situational  awareness  is  about 
the  ability  to  base  inferences  about  the  enemy  units  and  weapon  sys¬ 
tems  on  the  observed  CROP  and  other  individual  and  team  factors.1 
The  information  about  an  enemy  unit  is  expressed  in  terms  of  unit 
features :  prominent  parts  or  characteristics  of  the  unit.  For  example, 
one  prominent  feature  might  be  the  location  of  the  unit;  others 
would  be  its  identity  or  combat  strength.  We  collected  all  these  fea- 
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tures  into  a  set  and  refer  to  it  as  the  feature  vector,  denoted 
F- ...fn ]r>  where  each  f  represents  an  individual  feature. 
Collecting  all  the  feature  vectors  for  all  the  enemy  units  in  the  bat- 
tlespace  produces  a  matrix:  F  =  [Ff,F2, . . . ,  Fm\ .  For  our  purposes,  the 
CROP  comprises  estimates  of  the  features  in  this  matrix,  and  the 
degree  to  which  it  reflects  ground  truth  is  a  measure  of  its  quality. 
Making  inferences  about  the  units  then  consists  of  the  ability  to  “fill 
in  the  gaps  in  the  features.’’ 

The  process  Figure  2.2  depicts  is  sequential,  consisting  of  five  steps. 
Let  F,  be  a  matrix  of  the  CROP’S  estimates  of  the  feature  vectors  at  the 
end  of  the  zth  step.  The  quality  of  information  associated  with  the 
feature  matrix,  F,  at  the  output  of  the  zth  process  (z  &  1)  is  defined  as 
follows: 


Q(f,'.Fm  )  =  Q(F/_i  )  Q(f;-  |  F,_ , ) . 


where  0<  QIF,  .!)  <  lis  the  quality  of  information  in  Ff  _x;  Q(F;-,  F^)  is 
the  quality  of  the  information  at  the  output  of  the  zth  process,  based 
jointly  on  the  estimates  in  the  feature  matrices  in  the  last  step  and  in 
the  current  step;  and  Q(F,|F,_i)  is  the  transfer  function  calculating  a 
partial  value  of  Q(F,:)  given  a  specific  F . 

Note  that  Q(F,)  is  a  vector-valued  function,  with  entries  correspond¬ 
ing  to  a  particular  quality  of  information  measure  (completeness, 
correctness,  or  currency).  Thus,  the  product  shown  above  is  a  scalar 
(a  term-by-term  multiplication  of  elements),  not  a  matrix,  product. 

A  full  expansion  of  the  methodology  results  in  the  following  chained- 
product  transformations: 

Q(F0 , Fx , F2 , F3 )  =  q(f0 )  q(fx  |  F0 )  Q(F2  | F0 , Fl)  q(f3  |F0,F,,F2) 

A  =  /z(q(f0,  F1,F2,F3),F4) 

SA  =  g(A,F5), 


where  Q(F0,  F1(  F2,  F3)  is  the  quality  of  information  in  the  observed 
CROP.  Matrix  F0  is  ground  truth;  its  quality  is  therefore  taken  to  be  1 : 
Q(F0)  =  1.  A  is  the  degree  of  individual  awareness,  based  on  the  qual¬ 
ity  of  the  observed  CROP,  the  expertise  of  the  individual,  and  other 
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information  available  to  him.  From  this  point,  the  product-form 
model  no  longer  applies,  and  the  natures  of  the  functions  h  and  g  are 
included  in  the  development  of  the  metrics  in  the  cognitive  domain. 
Finally,  SA  is  the  measure  of  shared  situational  awareness,  which  is 
dependent  on  individual  awareness  and  the  characteristics  of  the 
collaborating  team. 

SUMMING  UP 

The  objective  of  this  research  is  to  develop  a  quantitative  methodol¬ 
ogy  that  allows  us  to  link  improvements  in  C4ISR  capabilities  to  their 
effects  on  combat  outcomes.  For  this  first  effort,  we  have  confined 
our  work  to  assessing  the  effects  of  data- collection  and  information- 
fusion  processes  and  the  dissemination  of  the  fused  CROP  on  indi¬ 
vidual  situational  awareness  and,  through  the  collaboration  process, 
on  shared  situational  awareness. 

We  use  the  Information  Superiority  Reference  Model  (Figure  2.1)  to 
describe  the  activities  associated  with  these  processes.  The  model 
consists  of  three  “domains”  extending  from  the  battlefield  environ¬ 
ment  to  cognitive  awareness  of  the  battlefield  situation  and  decision. 
The  transformation  functions  that  the  information  superiority  value 
chain  (Figure  2.2)  depicts  describe  the  value  of  these  processes  to 
combat  operations.  The  quantitative  methodology  is  based  on  these 
transformations. 

Enemy  battlefield  entities  (units  and  weapon  systems)  are  described 
in  terms  of  their  features  or  characteristics;  hence,  the  quality  of  the 
information  concerning  the  entities  is  an  assessment  of  how  well  the 
C4ISR  system  estimates  the  features  of  the  collected  set  of  enemy 
units  in  the  battlespace.  A  conditional  product-form  model  is  used  to 
measure  the  effects  of  the  value  chain  transformations  for  informa¬ 
tion  quality,  while  the  effect  of  information  quality  on  individual 
awareness  and  shared  situational  awareness  is  defined  through  a 
more  general  functional  relationship.  We  defer  the  detailed  discus¬ 
sion  of  the  last  set  of  functions  to  Chapter  Four. 


Chapter  Three 

THE  PHYSICAL  AND  INFORMATION  DOMAINS 


It  is  easier  to  perceive  error  than  to  find  truth,  for  the  for¬ 
mer  lies  on  the  surface  and  is  easily  seen,  while  the  latter 
lies  in  the  depth,  where  few  are  willing  to  search  for  it. 

— Johann  Wolfgang  von  Goethe 


The  physical  and  information  domains  are  the  sources  of  informa¬ 
tion  used  to  inform  decisions  in  the  cognitive  domain.  We  discuss 
them  together  in  this  chapter  because  of  their  close  connection:  The 
ground  truth  resident  in  the  physical  domain  is  approximated 
through  data  collection  and  processing  in  the  information  domain. 
The  observed  CROP  is  the  approximation  of  the  relevant  portions  of 
ground  truth  disseminated  to  friendly  users. 

THE  PHYSICAL  DOMAIN 

The  physical  domain  is  the  beginning  and  the  end  of  the  C4ISR 
cycle — both  the  subject  of  and  object  of  decision.  Chapter  Two 
described  this  domain  as  consisting  of  the  disposition  of  friendly  and 
enemy  forces  in  the  AO,  including  friendly  and  enemy  collection 
assets,  fusion  facilities,  networks,  and  command  and  control  facili¬ 
ties;  the  geospatial  features  of  the  region  in  which  the  forces  are 
engaged;  and  such  intangibles  as  enemy  intent,  plans,  and  current 
and  projected  activities.  Before  combat  begins,  information  about 
ground  truth  is  generally  available  through  the  IPB  process.  Infor¬ 
mation  about  some,  but  not  all,  elements  of  ground  truth  is  perish¬ 
able.  In  any  case,  the  friendly  commander’s  knowledge  about  all 
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aspects  of  ground  truth  is  uncertain.  Part  of  the  C4ISR  system’s  func¬ 
tion  is  to  reduce  that  uncertainty  as  the  operation  unfolds.  Once  a 
decision  has  been  reached  and  actions,  including  force  synchroniza¬ 
tion,  have  been  taken  to  ensure  its  implementation,  the  elements  of 
the  physical  domain  may  be  altered  in  a  way  that  may  not  be  known 
to  the  friendly  commander.  Thus  the  cycle  begins  anew. 

Features 

The  physical  domain’s  ground  truth  is  described  in  terms  of  the  fea¬ 
tures  of  the  entities  the  CROP  comprises.  For  our  purposes,  the 
CROP  consists  of  information  about  enemy  units  only.  Higher-level 
assessments  that  would  include  enemy  intentions  and  plans,  based 
on  patterns  and  other  information,  are  a  next  step.  A  feature  is  a 
prominent  part  or  characteristic  of  a  unit  that  is  relevant  to  the 
friendly  commander;  Table  3.1  offers  examples.  Enough  features 
must  be  collected  to  communicate  to  the  commander  the  current 
estimate  of  the  combat  situation.  Thus,  the  list  of  features  does  not 
necessarily  describe  all  the  units  in  the  physical  domain  completely. 
To  the  list  of  features  in  the  table,  we  could  add  the  destination  of  the 
force,  the  activity  it  is  currently  engaged  in,  its  echelon,  etc. 

The  Relevant  Ground  Truth 

Chapter  Two  characterized  the  relevant  ground  truth  as  consisting  of 
a  matrix,  F  =  [Fh  F2, ,  Fm],  where  each  Ft  represents  a  vector  whose 
elements  are  the  features  of  unit  i.  Our  previous  discussion  noted 

Table  3.1 

Some  Unit  Features 


Feature 

Description 

Location 

The  current  placement  of  the  unit  in  the  bat- 
tlespace,  using  x-y  coordinates 

Speed 

Rate  of  advance  to  friendly  positions,  in 
kilometers  per  hour 

Direction 

Azimuth,  in  radians 

Strength 

A  combat  effectiveness  score 

Unit  type 

The  predominant  military  organization  type 
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that  the  features  necessarily  vary  over  time,  for  example,  the  location 
of  a  moving  unit.  This  implies  that  matrix  F  is  time  dependent. 
Although  we  have  made  no  notational  adjustments  to  the  matrix,  it 
should  be  understood  that  it  is  dynamic.  In  addition,  we  track  the 
status  of  estimates  of  F  through  domains  by  assigning  subscripts  to 
indicate  where  we  are  in  the  process.  Accordingly,  the  matrix  in  the 
physical  domain  is  designated  F0.  In  the  physical  domain,  the  quality 
of  the  information  about  F0  is  considered  “perfect,”  in  that  it  is 
ground  truth,  so  that  Q(F0)  =  1.  Next,  in  the  information  domain,  the 
quality  of  the  information  in  the  CROP  reflects  how  well  it  represents 
the  relevant  ground  truth,  measured  in  terms  of  the  completeness, 
correctness,  and  currency  of  the  data  provided  by  the  sensor  suite 
and  how  well  the  data  have  been  processed  and  analyzed. 

THE  INFORMATION  DOMAIN 

Information  collection,  processing,  and  dissemination  take  place  in 
the  information  domain,  which  is  further  subdivided  into  three  sub- 
domains:  sensor  (intelligence,  surveillance,  and  reconnaissance), 
fusion,  and  network.  These  three  constitute  the  main  effort  of  the 
C4ISR  system  and  parallel  the  three  information  domain  graphs 
depicted  as  part  of  the  information  superiority  value  chain  in  Figure 
2.2.  The  sensors  collect  data  about  the  relevant  ground  truth  units 
from  the  physical  domain,  then  forward  it  to  fusion  facilities,  where  it 
is  processed,  fused,  and  eventually  shared  with  friendly  users 
through  a  communications  network.  Figure  3.1  depicts  the  transfor¬ 
mation  equations  from  Chapter  Two,  showing  the  transformations 
from  collection  in  the  sensor  subdomain  to  dissemination  in  the 
network  subdomain.  The  end  product  is  an  observed  CROP,  a  por¬ 
tion  of  which  is  available  to  each  friendly  user. 

THE  SENSOR  SUBDOMAIN 

Information  about  the  physical  domain  originates  with  the  sensors 
and  information  sources  allocated  or  directed  to  the  AO.  The  sensor 
array  is  generally  referred  to  as  the  sensor  suite.  Its  configuration  will 
vary  with  the  operational  situation  and  the  characteristics  of  the  sen¬ 
sors.  The  output  from  the  sensor  domain  is  a  set  of  feature  vectors, 
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Figure  3.1 — Information  Domain  Transformations 


F1(  organized  by  discipline  (such  as  imagery  intelligence  [IMINT], 
electronic  intelligence  [ELINT],  or  moving  target  indication  [MTI]]. 
The  quality  of  this  representation  of  the  CROP,  Q(F0,  Fi),  is  depen¬ 
dent  on  the  relevant  ground  truth,  depicted  as  F0  in  Figure  3.1,  and 
the  performance  of  the  sensor  suite. 

Following  a  short  discussion  of  sensors  and  their  characteristics,  we 
suggest  measures  for  the  completeness,  correctness,  and  currency  of 
the  sensor  reports  arriving  at  fusion  facilities. 

Sensors  and  Sources 

Sensors  are  designed  to  detect  objects,  record  images  of  designated 
areas,  and  estimate  physical  phenomena.  They  are  capable  of  per¬ 
forming  systematic  surveillance  and  reconnaissance  over  large  areas, 
subject  to  the  existence  of  threats  that  may  jeopardize  sensor  plat¬ 
form  survival.  Sensors  may  be  capable  of  detecting  types  or  classes  of 
militarily  relevant  objects  or  targets  or  may  detect  whole  classes  of 
objects,  such  as  moving  vehicles.  In  general,  sensor  performance  is  a 
function  of  the  environment  (terrain,  foliage,  electromagnetic  back¬ 
ground  noise,  extraneous  reflected  sunlight  or  glint,  etc.) .  For  exam¬ 
ple,  radar  sensors  that  operate  in  the  microwave  band  can  detect 
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only  targets  that  have  a  radar  cross  section  above  some  minimum 
threshold  and  that  are  in  environments  with  signal-to-noise  ratios 
above  some  minimum  threshold. 

Sources,  on  the  other  hand,  may  be  covert  and  typically  operate  over 
much  smaller  areas.  Sources  include  such  things  as  human  observa¬ 
tions;  very  short  range  communications  intercepts  or  surveillance; 
unattended  covert  devices  that  can  be  read  out  intermittently;  and  a 
priori  knowledge  about  enemy  force  dispositions,  future  plans,  etc. 

Active  and  Passive  Sensors 

Sensors  are  generally  either  active  or  passive.  Active  sensors  transmit 
a  signal  that  impinges  on  surrounding  objects  and  the  environment. 
An  active  sensor  also  receives  and  processes  the  reflection  of  this 
transmitted  signal.  Because  of  the  time  delay  associated  with  two- 
way  transmissions  of  information,  active  sensors  can  be  used  to 
determine  the  location  of  the  target  to  some  degree  of  accuracy. 
Examples  of  active  sensors  are  Doppler  radar,  ground  moving  target 
indicator  (GMTI)  radar,  and  laser  rangefinders. 

Passive  sensors  act  only  as  receivers  and  can  detect  a  target  only  if  it 
emits  some  sort  of  signal  or  if  it  emits  radiant  or  reflected  energy  at  a 
higher  rate  than  does  the  background  environment.  Examples  of 
passive  sensors  are  signals  intelligence  receivers;  electro-optical 
imaging  devices,  including  traditional  telescopes;  and  infrared  sen¬ 
sors. 

For  many  nonimaging  passive  sensors,  a  single  sensor  is  not  capable 
of  determining  the  location  of  a  target.  Two  or  more  nonimaging 
passive  sensor  target  detections  are  needed  to  reduce  target  location 
error  to  a  useful  level.  Thus,  the  fusion  process  is  important  for 
extracting  useful  surveillance  and  targeting  information  (and  not  just 
intelligence)  from  the  output  of  many  passive  sensors.  On  the  other 
hand,  because  they  make  use  of  the  original  emissions  of  the  target 
or  because  they  can  produce  high-resolution  images  of  the  target, 
passive  sensors  can  greatly  assist  in  identifying  or  classifying  targets. 
High-resolution  imaging  sensors  have  traditionally  been  most  valu¬ 
able  for  locating  and  classifying  targets.  However,  such  sensors  can 
be  large  and  expensive  and,  if  placed  on  airborne  platforms,  can  be 
vulnerable  to  attack. 
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Sensor  Detections 

Sensors  can  typically  detect  targets  or  objects  only  when  certain 
conditions  prevail.  For  example,  synthetic  aperture  radar  (SAR)  can 
detect  images  of  stationary  targets,  but  cannot  detect  moving 
objects.  Target  objects  must  be  larger  than  some  minimum  size  to  be 
resolved  in  a  SAR  image,  and  target  object  features  must  be  farther 
apart  than  the  imaging  resolution  of  the  SAR  imaging  sensor  for  the 
target  to  be  identified  or  classified.  Similarly,  a  GMTI  radar,  when 
operating  in  this  mode,  can  detect  moving  targets  but  cannot  detect 
stationary  targets.  In  addition,  it  can  detect  moving  targets  only  if 
they  are  moving  at  a  rate  between  some  minimum  and  some  maxi¬ 
mum  velocity.  Infrared  imaging  sensors  can  detect  a  target,  moving 
or  stationary,  only  if  its  temperature  is  above  some  minimum  relative 
to  the  temperature  of  the  background  environment. 

Military  operations  typically  use  a  wide  variety  of  sensors.  Each  has 
its  strengths  and  weaknesses.  The  generic  model  of  sensor  perfor¬ 
mance  in  this  analysis  is  composed  of  probabilities  of  detection  for 
sensors  as  a  function  of  range.  Target  location  and  velocity  errors 
(direction  and  speed)  are  also  expressed  as  a  function  of  range. 

Completeness 

The  information  gaps  in  the  sensor  subdomain  relate  to  the  number 
of  enemy  units  detected,  the  features  reported  for  the  units  detected, 
and  the  sensor  suite’s  coverage  area.  For  information  to  be  complete, 
all  features  of  all  units  in  the  relevant  ground  truth  must  be  known, 
and  the  entire  AO  must  be  under  sensor  observation.  This  suggests 
the  two  measures  and  associated  metrics. 

The  Detection  of  Units  in  the  AO.  It  is  assumed  that  the  total  number 
of  enemy  units  in  the  AO  is  known.  This  is  not  too  constraining  given 
the  IPB  process,  although  this  may  vary  by  command  echelon.  Dis¬ 
criminating  between  real  and  false  detections  is  a  more  serious 
problem.  False  detections  can  result  from  deception  or  from  the 
inability  of  the  sensor  suite  to  distinguish  between  military  and 
nonmilitary  systems.  GMTI  radars,  for  example,  will  detect  many 
objects  that  are  not  militarily  relevant  because  they  detect  all  ground 
vehicles  moving  above  a  certain  speed.  Reports  from  the  current 
generation  of  GMTI  sensors  must  be  transmitted  to  fusion  facilities 
(as  discussed  below)  to  distinguish  military  targets  from  other  similar 
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objects.  The  degree  to  which  an  individual  sensor  or  suite  of  sensors 
can  distinguish  military  targets  from  other  similar  objects  is  the  sub¬ 
ject  of  the  correctness  performance  measure  discussed  below.  What 
is  relevant  to  the  completeness  measure  is  the  degree  to  which  mili¬ 
tary  targets  are  not  detected. 

An  appropriate  metric  corresponding  to  this  measure  is  calculating 
or  estimating  the  percentage  of  enemy  units  in  the  AO  that  have  been 
detected.  Developing  an  estimate  requires  a  model  of  sensor  perfor¬ 
mance  and  a  model  of  the  specific  environment  in  which  the  sensor 
is  operating.  High-fidelity  models  of  sensor  performance  typically 
include  detailed  descriptions  of  the  environment  in  which  the  sensor 
operates  and  the  targets  the  system  is  designed  to  detect  and  classify. 
High-fidelity  models  are  applicable  only  for  specific  types  of  sensors. 
Thus,  modeling  the  overall  performance  of  a  complete  sensor  suite  at 
high  fidelity  requires  a  set  of  high-fidelity  sensor  models.  Such  an 
approach  is  beyond  the  scope  of  the  present  investigation  and  is  not 
appropriate  to  the  problem  at  hand.  Appendix  B  describes  a  generic 
sensor  model  that  can  be  calibrated  to  represent  the  performance  of 
a  wide  array  of  possible  sensors. 

Sensor  Suite  Coverage.  This  simple  calculation  is  based  on  the  col¬ 
lective  sensor  areas  of  regard  and  their  revisit  rates.  An  appropriate 
metric,  therefore,  is  simply  the  fraction  of  the  AO  covered.  Redun¬ 
dant  coverage,  however,  can  cause  the  metric  to  assume  values 
greater  than  1.0.  This  is  not  a  problem  because  this  metric  must  be 
combined  with  the  fraction  of  units  detected  to  produce  a  composite 
completeness  transfer  function. 

If  we  let  c1  and  c2  represent  the  metrics  for  the  fraction  of  units 
detected  and  the  fraction  of  area  covered,  respectively,  combining 
the  two  metrics  produces  the  completeness  component  of  the 
transfer  function  Q(F,  |F0)  depicted  in  Figure  3.1.  Clearly,  these  two 
metrics  are  not  independent.  Complete  or  even  redundant  area  cov¬ 
erage  makes  it  more  likely  that  a  larger  number  of  units  will  be 
detected  and  their  features  known.  Redundant  coverage  also  miti¬ 
gates  against  false  target  detections.  The  following  combining  func¬ 
tion  reflects  this: 


Qcom(Fl|Fo)  =  C1(1-e'C2) 
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With  this  formulation,  the  combined  completeness  score  is  always 
less  than  the  fraction  of  units  detected.  However,  as  the  coverage 
increases,  it  approaches  but  never  exceeds  that  fraction.  As  the  sen¬ 
sor  area  coverage  increases,  and  becomes  redundant,  the  fraction  of 
real  units  detected  more  closely  matches  the  total  number  detected. 
For  example,  if  70  percent  of  the  units  in  the  AO  have  been  detected 
(real  and  false)  and  if  the  area  coverage  is  300  percent  (that  is,  the 
sensor  suite  is  capable  of  effectively  sweeping  the  AO  three  times 
over),  this  combining  function  yields  an  estimate  of  the  fraction  of 
real  targets  detected  of  0.665.  We  chose  this  form  for  the  combining 
function  because  it  both  strongly  penalizes  completeness  if  not 
much  of  the  AO  is  sensed  (in  which  case,  we  might  consider  a  high 
unit-detection  score  as  being  lucky,  not  good)  and  accurately  indi¬ 
cates  that  there  is  little  marginal  benefit  in  increasing  sensor  cover¬ 
age  if  most  of  the  AO  is  already  being  “oversensed”  (for  example, 
increasing  sensor  coverage  from  400  to  401  percent  if  of  little  bene¬ 
fit). 

Correctness 

The  measures  and  metrics  for  correctness  focus  on  deviations  from 
ground  truth.  In  an  absolute  sense,  a  report  is  completely  correct  if  it 
matches  ground  truth  exactly.  For  example,  a  report  that  an  observed 
unit  is  mechanized  is  correct  only  if  the  observed  unit  is  indeed 
mechanized.  We  recognize  that,  in  some  contexts,  a  report  that  the 
unit  is  armored  would  be  a  “correct”  identification  if  the  only  con¬ 
cern  were  whether  the  unit  has  tracked  vehicles.  For  our  purposes, 
however,  we  have  taken  the  more-absolute,  context-free,  approach 
and  define  a  measure  of  correctness  to  be  the  degree  to  which  the 
true  target  features  approximate  their  ground  truth  values. 

There  are  several  ways  to  assess  the  nearness  of  an  estimate  to  its 
ground  truth  value  mathematically.  We  mention  two  here,  one  using 
estimation  theory  and  the  other  assessing  precision.  The  estimation 
theory  approach  is  appropriate  when  analysts  know  ground  truth 
directly  (as  in  simulations  or  controlled  experiments);  the  precision 
method  is  appropriate  when  analysts  do  not  know  ground  truth  (as 
in  real-world  operations). 

Estimation.  The  entire  sensor-collection  process  is  essentially  an 
attempt  to  estimate  the  ground-truth  values  of  enemy  unit  features, 
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intent,  plans,  and  activities.  It  therefore  seems  natural  to  suggest  that 
estimation  theory  might  be  useful  for  assessing  the  correctness  of  the 
estimate.  If  we  view  the  collection  of  data  about  the  enemy  unit  fea¬ 
tures  as  a  collection  of  random  samples,  we  can  calculate  an  estimate 
of  the  true  value  of  the  feature  using  one  of  several  statistical  estima¬ 
tors.  For  example,  suppose  we  wish  to  estimate  the  location  of  an 
enemy  unit  based  on  several  sensor  reports.  Assuming  that  the  loca¬ 
tion  of  the  unit  will  be  reported  as  an  x-y  coordinate,  we  need  an 
estimate  for  both  the  x-coordinate  and  the  y-coordinate.  The  sim¬ 
plest  estimator  is  the  sample  mean.  If  X,-  =  [xityj\  represents  the  zth 
sensor  report  estimate  for  the  enemy  unit,  the  following  is  a  simple 
estimate  of  its  true  location: 


ji  =  — £"=i  X,: , 

where  n  is  the  total  number  of  reports  arriving. 

The  chore  now  is  to  assess  just  how  correct  this  estimate  is.  If  we 
were  conducting  a  controlled  experiment,  such  that  the  true  location 
of  the  enemy  unit  is  known,  then  we  could  take  advantage  of  the  fact 
that  an  unbiased  estimator  is  one  in  which 

where  p  =  [/rx,py]  is  the  true,  known,  mean  value.  For  our  purposes, 
we  can  take  this  to  be  the  true  location  of  the  unit.  The  correctness  of 
the  estimate  in  each  direction  can  be  calculated  to  be 

a  =  |e[p]-p|  , 

generally  referred  to  as  the  bias  in  the  estimate.1  Calculating 

e[p 


1Note  that  this  refers  to  statistical  bias,  not  operational  bias.  We  assume  that  the  esti¬ 
mates  of  features  have  been  corrected  for  sensor  bias. 
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requires  several  samples  of  sensor  reports  or  the  partitioning  of  a 
single  sample.  For  nonquantitative  features,  the  mode  of  all  reports 
might  be  used  as  the  estimator.  However,  calculating  the  bias  is 
problematic.  One  technique  is  to  develop  similarity  matrices.2  One 
way  to  use  the  bias  to  calculate  the  correctness  component  of  the 
transformation  function  in  Figure  3.1  is 

Qcor(Fl  |  F0 )  =  e~(Ax+Ay}  , 

where  Ax  and  Ay  are  the  biases  in  the  x-  and  y-position  estimates, 
respectively.  If  we  take  the  estimate  to  be  completely  correct  when 
Qcor(Fi|F0)  =  1,  correctness  gets  worse  for  large  biases,  as  expected. 
The  use  of  the  exponential  form  rapidly  penalizes  correctness  as  bias 
errors  initially  increase;  correctness  levels  out  as  the  biases  become 
large  enough  to  impair  mission  effectiveness,  such  that  any  addi¬ 
tional  bias  has  little  effect. 

Several  estimators  other  than  the  sample  mean  might  be  used.  One 
of  these,  the  linear  estimator,  is  a  linear  function  of  the  sensor  obser¬ 
vations.  For  example,  we  can  estimate  the  location  of  a  moving 
enemy  unit  at  some  time  in  the  future  by  observing  its  current  loca¬ 
tion  and  its  velocity.  Multiplying  the  x-  and  y-components  of  the 
velocity  by  the  time  elapsed  added  to  the  current  estimate  makes  the 
location  of  the  enemy  unit  a  linear  function  of  its  velocity.  Maximum 
likelihood  and  minimum  mean-square  estimators  can  also  be  used, 
as  appropriate.3 

Precision.  Bias  appears  to  be  a  good  way  to  measure  correctness 
when  conducting  an  experiment  in  which  the  true  value  of  the  fea¬ 
ture  to  be  estimated  is  known.  However,  assessing  the  correctness  of 
sensor  observations  when  the  true  values  of  the  features  are  not 
known  (as  in  support  of  combat  operations)  requires  using  other 
methods.  One  possibility  is  to  calculate  the  precision  of  the  reports. 
Precision  is  the  ability  of  a  sensor  suite  to  provide  repeated  estimates 
that  are  very  close  together.  This  is  another  way  of  saying  that  some 


2Appendix  B  offers  a  methodology  for  producing  such  a  matrix. 

3Any  standard  statistics  text  can  provide  more  details  on  these  estimators.  See,  for 
example,  Stark  and  Woods  (1986). 
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function  of  the  variance  in  the  sample  of  estimates  might  serve  as  a 
metric.  Continuing  the  above  example,  suppose  now  that  the  ground 
truth  location  of  the  enemy  units  is  not  known  and  that  the  sensors 
reported  27  locations  for  enemy  units,  as  depicted  in  Figure  3.2. 
Using  a  suitable  cluster  algorithm  indicates  that  the  reports  are 
widespread  enough  to  suggest  that  there  are  four  enemy  units  in  the 
observed  area.4  The  algorithm  produces  a  cluster  representative  loca¬ 
tion,  depicted  as 


X,: 

in  the  diagram.  The  cluster  representatives  can  then  be  taken  to  be 
the  location  estimates  for  the  enemy  units. 
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Figure  3.2 — Measuring  Precision 


Clustering  is  used  when  no  prior  information  is  available  on  the  disposition  of  the 
enemy  forces.  With  prior  information,  the  observations  “cluster”  around  that  previous 
information.  For  a  complete  discussion  of  cluster  algorithms,  see  Gordon  (1999)  and 
Duda  and  Hart  (1973). 
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The  location  of  each  enemy  unit  is  taken  to  be  a  bivariate  normal 
random  variable  whose  mean  is  the  ground  truth  location  of  the  unit 
and  whose  covariance  is  estimated  by  the  following  sample 
covariance  matrix: 


In  this  formulation, 


Xj  =  [ x/-y/ 1  represents  the  other  cluster  locations,  and  S2is  the  sample 
variance  in  both  the  x-  and  y-directions.  The  zeroes  in  the  off- 
diagonal  positions  reflect  the  fact  that  the  x-  and  y-positions  of  the 
enemy  unit  are  independent.  Precision  is  then  defined  to  be  the 
determinant  of  the  covariance  matrix: 


p=  i=s4. 


Note  that  this  value  is  always  nonnegative  and  that  a  0  value  implies 
perfect  precision.  Therefore,  the  correctness  component  of  the 
transformation  function  in  Figure  3.1  might  be  Qcor(Fi|F0)  =  e  If  we 
continue  the  same  convention  as  with  the  bias  metric — that  com¬ 
plete  correctness  occurs  when  Qcor(FilFo)  =  1 — correctness  gets  worse 
for  large  values  of  p.  Since  precision  is  exact  when  p  =  0,  this  pro¬ 
duces  the  desired  effect. 

Currency 

In  general,  we  assumed  that  the  less  time  required  to  complete  a 
process  the  better.  Therefore,  currency  measures  generally  assess  the 
time  required  to  perform  functions.  For  the  sensor  subdomain,  the 
measure  is  the  time  required  to  complete  sensor  operations  and  local 
data  processing.  The  number  of  tasks  this  requires  clearly  depends 
on  the  situation.  Some  may  be  accomplished  in  parallel,  while  others 
might  be  sequential.  In  any  case,  what  is  required  is  an  estimate  of 
the  total  time  elapsed  between  detection  and  receipt  of  the  report  at 
the  fusion  center. 
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Suppose  two  tasks  consume  time  for  a  given  sensor  suite: 

•  completing  target  detection  and  establishing  a  target  track  for  an 
enemy  unit 

•  retasking  sensors  to  provide  additional  coverage  of  units  to 
reduce  uncertainty  in  the  estimates. 

The  total  time  expended  in  getting  a  set  of  reports  to  the  fusion  cen¬ 
ter  is  dictated  by  the  report  that  takes  the  longest.  The  methodology 
used  to  assess  this  time  is  referred  to  as  the  critical  path  method. 
Because  this  same  method  is  used  for  all  currency  calculations, 
details  are  included  in  Appendix  B. 

Suppose  that  the  time  required  to  get  a  report  containing  several 
observations  based  on  initial  detections  and  retaskings  to  the  fusion 
facility  is  t.  In  the  absence  of  any  other  situation-dependent  infor¬ 
mation,  we  assume  that  currency  is  greater  for  smaller  values  of  t,  so 
that  a  task  that  is  accomplished  instantaneously  has  maximum  cur¬ 
rency.  Applying  logic  similar  to  that  applied  to  the  completeness  and 
correctness  metrics  yields  a  currency  component  of  the  transforma¬ 
tion  function  in  Figure  3.1  of  Qcur(F|  |F0)  =  e In  addition  to  mapping 
the  reporting  time  to  a  0-1  value  as  desired,  with  1  being  best,  using 
the  exponential  function  heavily  emphasizes  completing  the  report 
quickly  while  not  greatly  distinguishing  between  lengthy  reporting 
times. 

THE  FUSION  SUBDOMAIN 

The  output  of  the  sensor  subdomain  is  a  series  of  sensor  reports  that 
are  forwarded  to  fusion  centers  for  processing.  For  purposes  of  this 
study,  we  assumed  that  reports  from  like  and  disparate  sensors  and 
sources  would  be  combined  to  produce  a  CROP,  which  would  subse¬ 
quently  be  disseminated  to  friendly  users.  The  quality  of  the  infor¬ 
mation  the  process  produces  is  represented  by  Q(F0,F1,F2)  in  Figure 
3.1.  The  correctness  and  completeness  of  the  process  are  condi¬ 
tioned  on  the  quality  of  the  information  and  data  received  from  the 
sensor  suites,  represented  by  Q(F0,F!)  in  Figure  3.1. 

Fusion 

Fusion  is  the  process  of  combining  information  from  sensors  and 
sources  to  produce  a  common,  relevant  picture  of  the  battlespace.  It 
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includes  the  correlation  and  analysis  of  data  inputs  from  supporting 
sensors  and  sources.  The  relevant  picture  drawn  can  vary  depending 
on  how  the  fused  information  is  to  be  used.  If,  for  example,  the 
information  is  to  be  used  only  for  targeting,  simply  combining 
reports  to  pinpoint  the  location  of  enemy  targets  is  sufficient.  If  the 
information  is  to  be  used  to  develop  plans  for  maneuver,  the  reports 
must  include  some  assessment  of  enemy  intent,  plans,  and  activities. 
Either  use  ultimately  leads  to  a  decision.  In  the  first  case,  targeting 
decisions  are  generally  made  at  the  tactical  level;  in  the  second, 
maneuver  decisions  are  generally  made  at  the  strategic  and  opera¬ 
tional  levels  of  combat. 

This  suggests  a  fusion  taxonomy  that  focuses  on  the  degree  to  which 
the  fusion  process  produces  inferences.  Table  3.2  lists  five  levels  of 
fusion  that  constitute  a  spectrum  of  fusion.  The  lower  levels  (0  and  1) 
represent  what  is  generally  included  in  the  CROP  today.  It  consists 
mainly  of  individual  target  locations  and  refined  tracks  posted  on  an 
area  map.  No  attempt  is  made  to  infer  unit  affiliation,  enemy  posture 
or  intent.  This  level  of  fusion  is  generally  sufficient  to  support  target 
acquisition,  but  its  contribution  to  situation  assessment  is  minimal. 
At  the  higher  levels  of  fusion,  fewer  burdens  are  placed  on  the  deci¬ 
sionmaker  in  that  he  is  not  as  required  to  draw  inferences  about 
capabilities  and  intent.  Automation  at  these  levels  is  rare  and  there¬ 
fore  when  it  is  done  at  all,  it  is  usually  a  manual  process  conducted 
by  the  unit  intelligence  element. 

Table  3.2 

Fusion  Processing  Levels 


Processing 

Level 

Description 

0 

Normalizing,  formatting,  ordering,  and  com¬ 
pressing  input  data 

1 

Refining  position,  tracks,  and  identity  estimates 

2 

Interpreting  relationships  between  objects  and 
events;  developing  situation  estimates 

3 

Assessing  enemy  capability  and  intent 

4 

Continuously  refining  estimates;  identifying 
needs  for  additional  sources  and  processing 

SOURCE:  These  definitions  were  adapted  from  Keithley  (2000). 
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As  mentioned  earlier,  however,  this  study  is  restricted  to  those 
aspects  of  ground  truth  that  focus  on  the  relevant  features  or  charac¬ 
teristics  of  enemy  units  and  not  on  enemy  intent,  plans,  and  activi¬ 
ties.  In  that  sense,  therefore,  the  fusion  we  are  dealing  with  is  at 
levels  0  and  1. 

Uses  of  Fused  Information 

The  lower  levels  of  fusion  are  generally  characterized  by  automated 
systems,  such  as  the  Army’s  Q-37  Firefinder  radar  system.  The  Q-37 
detects  enemy  artillery  and  mortar  fires  and  relays  the  information  to 
the  fire  control  center  within  seconds.  Counterbattery  fires  are  then 
launched  at  the  enemy  weapon  systems.  Sensors  that  detect  mobile 
missile  launchers  pass  their  information  to  orbiting  combat  air  patrol 
aircraft,  which  then  attempt  to  destroy  the  launchers.  The  level  of 
fusion  in  these  cases  is  rather  low  and  is  generally  completed  by  the 
sensor  system  itself.  For  example,  an  MTI  combines  location,  speed, 
and  direction  of  movement  to  help  establish  a  track  for  the  observed 
target.  This  might  apply  to  tracking  the  moving  missile  launcher. 

Assessing  the  enemy  situation  at  the  higher  levels  of  fusion  is  more 
complex,  generally  consisting  of  both  automated  and  manual  pro¬ 
cesses.  Much  more  information  may  be  needed  than  that  required 
for  target  nomination.  For  example,  it  may  not  be  enough  to  know 
where  the  enemy  is;  it  may  also  be  necessary  to  know  what  he 
intends  to  do  next  or  what  he  is  capable  of  doing  next.  These  assess¬ 
ments  generally  take  place  in  the  command  operating  center  as  an 
adjunct  to  the  intelligence  process.  However,  both  distributed  and 
centralized  systems  are  possible.  To  provide  input  to  what  is  in 
essence  a  manual  process,  the  Army  has  developed  and  imple¬ 
mented  the  All  Source  Analysis  System,  a  centralized  fusion  center 
where  data  from  all  sources  are  combined,  correlated,  analyzed,  and 
disseminated  to  eligible  users.  Other  systems  have  also  provided 
input  to  the  process.  In  the  mid-1980s,  U.S.  Air  Forces  Europe 
initiated  the  Enemy  Situation  Correlation  Element  to  do  the  same 
thing.  At  a  lower  level,  the  loint  Surveillance  Target  Advance  Radar 
System  (STARS)  Ground  Station  will  include  the  joint  STARS  Work 
Station,  an  automated  MTI  fusion  system.  None  of  these  systems  in 
itself  provides  higher  levels  of  fusion.  This  is  generally  performed  by 
the  command  intelligence  staff. 
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Fusion  Facilities 

Fusion  is  essentially  a  parallel-sequential  process.  Each  intelligence 
discipline  (INT)  attempts  to  fuse  its  internal  estimates  and  forward 
them  to  a  central  fusion  processor,  which  combines  fused  feature 
vectors  from  disparate  INTs  to  develop  the  CROP.  Although  this  may 
not  be  the  physical  model  in  all  cases,  the  basic  sequence  is  generally 
applicable.  Figure  3.3  illustrates  the  process. 

It  is  generally  reasonable  to  assume  that  the  longer  a  fusion  facility 
has  to  examine  the  sensor  or  within- discipline  reports,  the  more  reli¬ 
able  the  results  will  be.  This  assumption  is  based  on  the  possibility 
that  the  longer  the  fusion  facility  has  to  complete  its  processing,  the 
more  information  will  be  available.  Also,  the  results  of  some  time¬ 
intensive  activities,  such  as  image  processing,  will  improve  given 
more  processing  time.  This  also  accounts  for  the  time  required  to 
retask  or  cue  sensors  to  focus  on  targets  of  interest. 

Automation  and  Control 

The  rate  at  which  the  center  classifies  detections  depends  in  part  on 
the  degree  of  automation  at  the  fusion  facility.  The  minimum  frac- 
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Figure  3.3 — Parallel-Sequential  Fusion  Process 
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tion  of  detections  that  can  be  classified  depends  on  the  characteris¬ 
tics  of  the  sensor  suite,  but  the  maximum  depends  on  the  ability  of 
the  fusion  center  to  control  the  supporting  sensor  suite.  We  treat 
automation  as  a  binary  variable:  Either  the  facility  is  automated  or  it 
is  not.  We  distinguish  two  types  of  operational  control:  dynamic  and 
static. 

Static  Control.  If  the  fusion  facility  can  only  control  the  sensors  by 
providing  their  initial  tasking,  it  has  static  control  over  the  sensors.  A 
sensor  management  plan  is  established  at  the  outset  of  operations, 
and  it  essentially  remains  fixed.  This  has  the  effect  of  slowing  the 
classification  process  if  the  plan  was  deficient  in  any  way.  Therefore, 
the  fraction  of  classified  detections  rises  slowly  over  time. 

Dynamic  Control.  If  the  fusion  facility  can  task  and  retask  the  sen¬ 
sors  to  confirm  reports  or  to  bridge  gaps  in  the  data,  it  has  dynamic 
control  over  the  sensors.  As  with  static  control,  the  fusion  facility 
begins  with  a  sensor  management  plan  but,  in  this  case,  can  alter  the 
plan  as  the  operation  progresses. 

Completeness 

The  ultimate  goal  of  the  fusion  process  is  to  create  a  picture  of  the 
battlefield  that  is  complete,  accurate,  and  current.  The  “picture”  con¬ 
sists  of  sets  of  estimated  values  for  the  features  of  enemy  units.  The 
objective  of  the  process  is  to  classify  the  unit  estimates  by  describing 
the  units  in  terms  of  the  relevant  features.  The  completeness  portion 
of  the  quality  measure,  then,  focuses  on  the  number  of  these  units 
contained  in  the  CROP. 

We  recognize  that  some  degree  of  classification  can  take  place  at  the 
sensor  level.  For  example,  cueing  an  unmanned  aerial  vehicle  to  fly 
over  a  location  to  view  a  detection  made  by  Joint  STARS  is  an  attempt 
to  classify  the  detected  object  using  only  a  sensor.  Assuming  this  has 
been  accomplished  for  one  or  more  of  the  targets,  the  chore  at  the 
fusion  facility  is  easier,  in  that  the  ability  to  “classify”  has  been 
enhanced.  Although  classification  can  take  place  at  both  the  sensor 
and  the  fusion  facilities,  we  chose  to  evaluate  its  quality  at  the  fusion 
center  to  clearly  delineate  functions. 

Classification  Rate.  The  rate  at  which  a  fusion  facility  can  classify 
detected  units  greatly  influences  the  number  of  detections  the  facil¬ 
ity  can  classify  in  a  fixed  period.  This  depends  on  several  factors  that 
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are  certainly  situation  dependent.  However,  it  may  be  possible  to 
resolve  all  these  factors  into  a  simple  expression,  c  =/(f).  That  is,  the 
fraction  of  detections  that  can  be  classified,  c,  is  dependent  on  the 
time  available,  t.  One  such  representation  is  an  increasing  exponen¬ 
tial.  Each  of  the  within-discipline  fusion  centers  and  the  central 
fusion  center  are  assigned  a  time-dependent  fraction  of  detections 
classified  of  the  form 


c  =  a+a(l-e  of| , 

where  a  represents  the  fraction  of  the  detected  targets  that  the  sen¬ 
sors  themselves  can  fuse;  a  +  a,  for  0  <  a  +  a  <  1,  is  the  maximum  frac¬ 
tion  of  the  detections  capable  of  being  classified  at  the  fusion  center; 
and  o  is  the  rate  at  which  detections  are  classified.  Verifying  that  the 
functional  relationship  is  appropriate  and  obtaining  realistic  values 
for  the  parameters  can  be  formidable. 

To  illustrate,  Figure  3.4  provides  several  examples.  For  each  curve,  a 
and  a  are  fixed  at  0.2  and  0.7,  respectively,  and  only  the  rate  of  clas¬ 
sification,  a,  changes.  Note  that,  in  this  example,  the  fraction  of 
classified  detections  for  all  centers  is  bounded  between  a  =  0.2  and 
a  +  a  -  0.9. 

Automation  and  Control  Effects.  Combinations  of  automation  and 
control  protocols  can  affect  the  classification  rate  at  each  of  the 
fusion  facilities.  The  effects  are  therefore  reflected  in  the  values  of  the 
parameters  a,  a,  and  o.  Assuming  that  the  functional  relationship 
between  the  fraction  of  detections  classified  and  the  time  is  correct, 
it  follows  that  automation  and  control  will  greatly  affect  the  shape  of 
the  curves  depicted  in  Figure  3.4. 

The  overall  completeness  of  the  fusion  process  for  k  INTs  can  there¬ 
fore  serve  as  a  reliability  model.  The  facilities  depicted  in  Figure  3.3 
process  estimates  in  parallel  and  feed  their  results  to  the  central 
processor  in  series.  If  we  assume  (1)  that  the  time  available  to  pro¬ 
cess  reports  at  each  of  the  fusion  facilities  is  fixed,  (2)  that  the  detec¬ 
tions  each  fusion  facility  processes  are  randomly  and  independently 
selected,  and  (3)  that  a  report  on  a  detection  from  a  single  fusion 
facility  is  necessary  and  sufficient  to  classify  that  detection  at  the 
central  processing  facility,  the  following  provides  a  measure  of  com¬ 
pleteness  for  the  simple  configuration  depicted  in  Figure  3.3: 
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Figure  3.4 — Fusion  Center  Classification  Rates 


C  = 


where  c,  is  the  fraction  of  the  detections  that  fusion  facility  i  can  pro¬ 
cess  in  the  time  available,  and  cc  is  the  same  quantity  for  the  central 
processing  facility.  This  quantity,  then,  is  taken  to  be  the  complete¬ 
ness  component  of  the  transformation  function  depicted  in  Figure 
3.1:  Qcom(p2|Fo>Fi)  =  C- 


Correctness 

As  in  the  sensor  subdomain,  the  appropriate  measure  of  correctness 
is  how  close  the  fused  estimate  for  each  enemy  unit  feature  is  to 
ground  truth — how  accurate  the  classifications  of  the  detections 
reported  are.  As  before,  the  problem  is  assessing  how  good  the  esti¬ 
mate,  and  therefore  the  fusion  process,  is.  However,  in  this  subdo¬ 
main,  we  added  the  additional  task  of  tracking  enemy  units.  This 
suggests  two  measures:  (1)  how  well  the  fused  features  of  the  units 
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reflect  ground  truth,  i.e.,  how  well  detections  can  be  classified,  and 
(2)  how  well  the  fusion  system  tracks  enemy  units  over  time. 

Classification.  Sensor  reports  arriving  at  fusion  facilities  may  or  may 
not  confirm  previous  reports.  Confirming  evidence  tends  to  reduce 
uncertainty,  and  disconfirming  evidence  tends  to  increase  it.  In 
either  case,  the  variation  among  the  reports  appears  to  be  central  to 
measuring  how  well  the  fusion  process  can  reflect  ground  truth.  With 
minimal  disconfirming  reports  arriving  concerning  the  features  of 
the  detected  units,  the  variance  in  the  estimates  will  likely  be  small, 
therefore  increasing  the  likelihood  that  a  “correct”  classification  will 
result  from  the  fusion  process.  The  opposite  is  true  if  the  number  of 
disconfirming  reports  is  large. 

This  suggests  that  one  way  to  address  this  problem  might  be  to 
examine  the  variance  in  the  feature  estimates  for  each  reported  unit. 
For  example,  suppose  the  fusion  process  is  to  classify  enemy  units 
based  on  three  reported  features:  location,  velocity,  and  unit  type. 
Therefore,  in  this  case,  correctness  measures  how  close  estimates  of 
these  three  features  are  to  ground  truth  for  each  enemy  unit  included 
in  the  reports.  Further  suppose  that,  for  a  given  unit,  there  are  10 
reports  containing  estimates  for  all  three  features.  Location  and 
velocity  are  bivariate  (x-  andy-position  for  location,  and  speed  and 
direction  for  velocity);  therefore,  the  determinant  of  the  sample 
covariance  matrices,  |2,|  and  |Z„|  for  location  and  velocity,  respec¬ 
tively,  would  be  a  measure  of  the  variance  among  the  estimates.  This 
is  similar  to  our  previous  assessment  of  precision  for  sensor  reports. 
The  estimates  of  unit  type  are  a  bit  more  problematic,  in  that  the 
types  are  nominal.  Nevertheless,  a  similarity  matrix  can  provide  an 
estimate  of  the  variance,  as  described  in  Appendix  B. 

This  approach  still  requires  two  more  levels  of  combination:  a  single 
measure  of  overall  variance  for  each  enemy  unit  reported  and  a  mea¬ 
sure  for  the  variance  among  all  the  reported  units.  Both  require 
combining  disparate  information.  Fortunately,  the  sample  variance 
is  always  positive.  Therefore,  if  we  create  the  metric  at  -  e  s\  where  s, 
is  the  sample  standard  deviation  for  the  ith  feature  of  the  unit,  a,  is 
close  to  1  for  small  variances  (better  correctness) .  For  large  variances, 
a j  approaches  0.  A  simple  weighting  scheme  might  then  lead  to  a  unit 
composite  score.  In  our  simple  example,  this  might  yield 

Wj  =  2  j=\  co  j  ci  j , 
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where 


rU  mi=1- 

This  produces  a  quantity  between  0  and  1,  where  a  value  close  to  1  is 
desirable.  The  problem  is  the  basis  for  selecting  the  weights.  One  cri¬ 
terion  might  be  the  relative  importance  of  the  feature  in  targeting  the 
enemy  unit. 

We  might  do  something  similar  for  the  next  combining  level,  assess¬ 
ing  the  effects  of  report  variances  among  all  the  enemy  units  in  the 
reports.  Establishing  the  weights  here  may  also  be  a  bit  problematic. 
This  requires  judging  the  relative  importance  of  classifying  the  vari¬ 
ous  enemy  units.  Nevertheless,  if  it  can  be  done,  we  would  again 
have  a  metric, 


w-2U  wtwit 

where 

2Uwi  =  i- 

This  also  produces  a  quantity  between  0  and  1.  This  quantity  then 
can  be  taken  to  be  the  classification  portion  of  the  correctness  com¬ 
ponent  of  the  transformation  function  depicted  in  Figure  3.1: 
Qcor,  l(F2|Fo-Fl)  =  W. 

Tracking.  Much  has  been  written  about  tracking,  notably  by  Samuel 
Blackman  in  a  survey  of  the  subject  he  published  in  1986.  Blackman 
was  interested  in  multiple-target  tracking,  which  is  applicable  to 
tracking  the  movement  of  enemy  units  in  the  battlespace.  Tracking  is 
recursive  in  that  observations  or  detections  of  enemy  units  must  be 
correlated  over  time.  The  basic  elements  of  a  recursive  tracking  sys¬ 
tem  are  depicted  in  Figure  3.5. 

For  this  example,  we  assumed  that  information  is  available  from  a 
previous  collection  cycle,  referred  to  as  a  scan.  First,  the  input  is 
examined  to  see  if  it  correlates  with  existing  tracks.  The  gating  func¬ 
tion  is  a  coarse  assessment  to  determine  whether  the  detection  is  a 
candidate  for  a  track  update  or  is  the  basis  for  a  tentative  new  track. 
If  the  former,  a  more  refined  detection-to-track  pairing  algorithm  is 
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SOURCE:  Adapted  from  Blackman  (1986)  p.  5. 


Figure  3.5 — Basic  Elements  of  a  Recursive  Tracking  System 


used  to  make  the  final  pairing.  A  tentative  track  is  confirmed  if  the 
number  and  quality  of  the  detections  meet  established  criteria.  A 
track  that  is  not  updated  becomes  degraded  and  must  be  deleted. 
Filtering  incorporates  the  correlating  detections  to  create  an  updated 
estimate.  Prediction  extrapolates  the  track  parameters  to  the  next 
scan.  If  no  update  is  available,  the  previous  update  is  used. 

The  correctness  of  the  tracks  of  enemy  units  that  this  or  a  similar 
process  produces  can  be  measured  in  terms  of  the  number  of  previ¬ 
ous  tracks  that  have  been  confirmed  during  this  scan  and  the  num¬ 
ber  of  new  tracks  initiated.  For  simplicity,  we  assumed  that  tracks  are 
never  deleted  but,  rather,  are  placed  in  an  inactive  status. 

If  tracks  exist  for  k  enemy  units  current  to  scan  t-  1  and  n  exist  at  the 
current  scan,  t,  we  must  examine  two  cases:  n  -  k  and  n>  k.  A  third, 
the  deleted-tracks  case  ( n  <  k),  would  not  apply  given  our  assump¬ 
tion  that  deleted  tracks  are  simply  put  in  an  inactive  status.  If  we  let 
Xr_x  represent  the  set  of  tracked  enemy  units  at  scan  t  -  1  and  Xf  the 
set  of  tracked  enemy  units  at  scan  t,  we  have  the  following  two  cases: 

a.  If  n  -  k,  we  either  are  able  to  correlate  the  tracks  in  XM  with  the 
tracks  in  Xf  or  are  not  able  to  do  so.  Three  possibilities  arise,  as 
depicted  in  Figure  3.6. 

b.  If  n  >  k,  there  are  more  tracks  at  scan  t  than  at  scan  t-  1.  This 
suggests  the  three  cases  depicted  in  Figure  3.7. 
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These  two  cases  allow  us  to  specify  the  fraction  of  the  enemy  units 
tracked  at  scan  t-  1  that  are  correlated  with  detections  at  scan  t  as  a 
metric  to  measure  how  well  the  system  is  able  to  track  enemy  units 
over  time.  In  Figures  3.6  and  3.7,  T  =  1.0  for  A  and  D;  T=  0  for  B  and  E; 
and  0  <  T<  1  for  C  and  F.5 

RAND  MR1467-3.6 

A  B  C 


Best  case:  All  enemy 
units  tracked  at  f-1 
correlate  with  all  the 
enemy  units  reported 
in  scan  t 


Worst  case:  No  enemy 
units  tracked  at  f-1 
correlate  with  enemy 
units  tracked  in  scan  f. 


Mixed  case:  Some 
enemy  units  tracked  at 
f-1  correlate  with  enemy 
units  tracked  in  scan  f 


Figure  3.6 — Tracking  Cases  When  n  =  k 


RAND  MR1467-3.7 


Best  case:  All  enemy 
units  tracked  at  f-1 
correlate  with  all  the 
enemy  units  reported 
in  scan  f. 


Worst  case:  No  enemy 
units  tracked  at  f-1 
correlate  with  enemy 
units  tracked  in  scan  f. 


Mixed  case:  Some 
enemy  units  tracked  at 
f-1  correlate  with  enemy 
units  tracked  in  scan  f 


Figure  3.7 — Tracking  Cases  When  n  >  lc 


5Appendix  B  includes  a  set  theoretic  discussion  of  this  metric. 
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This  quantity,  then,  can  be  taken  to  be  the  tracking  portion  of  the 
correctness  component  of  the  transformation  function  depicted  in 
Figure  3.1:  QCOr, 2(F2|F0,F1)  =  T.  We  might  combine  the  two  using  an 
importance  weight,  0  <  cn  <  1,  so  that  the  correctness  component  of 
the  transformation  function  is  QfW(F2|F0,F|)  =  to W+  (1  -  oj) T. 

Currency 

Currency  can  be  defined  as  the  total  time  required  to  update  the 
CROP  using  inputs  from  the  integrated  multisensor  suite.  If  t  is  the 
total  time  required  to  update  the  CROP,  Qcur(F2|F0,F|)  =  e~(  provides  a 
metric  for  the  currency  component  of  the  transformation  function 
depicted  in  Figure  3.1.  Instantaneous  updating  earns  a  score  of  1.0. 

THE  NETWORK  SUBDOMAIN 

There  are  two  networks  to  contend  with:  the  communications  net¬ 
work  that  supports  the  multisensor  system  and  the  network  that 
supports  the  dissemination  of  the  CROP  to  required  users.  The  latter 
is  the  network  subdomain  we  discuss  here,  through  which  the  CROP 
developed  in  the  fusion  subdomain,  F2,  is  disseminated  via  a  com¬ 
munications  network  that  connects  all  users  to  the  fusion  facilities. 
This  product,  the  observed  CROP,  informs  both  the  commander’s 
situational  awareness  and  the  decisions  he  is  to  take.  The  quality  of 
the  observed  CROP  is  conditioned  on  the  information  transmitted 
from  the  fusion  facilities. 

Communications  Networks 

The  battlefield  of  the  future  is  likely  to  be  highly  dispersed,  and  com¬ 
bat  will  therefore  be  nonlinear.  This  places  considerable  demands  on 
communications  networks  that  support  C4ISR  functions.  For  exam¬ 
ple,  is  it  more  efficient  to  create  a  single,  perhaps  out-of-area,  fusion 
center,  or  are  distributed  centers  more  efficient?  If  there  is  a  robust 
reachback  capability,  it  can  be  argued  that  considerable  efficiencies 
are  possible  if  fusion  resources  are  concentrated  at  a  single  site.  But 
this  takes  the  responsibility  for  developing  the  CROP  out  of  the 
hands  of  the  local  commander  and  can  easily  foster  a  “not  invented 
here’’  attitude.  Indeed,  this  occurred  during  the  Kosovo  conflict.  The 
Commander  of  Task  Force  Hawk  rejected  the  opinion  of  the  Joint 
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Assessment  Center  at  Molesworth,  England,  that  the  threat  from  the 
2nd  Yugoslav  Army  in  Montenegro  was  minimal  in  favor  of  his  own 
that  this  army  posed  a  threat  to  his  Apache  helicopters  based  at 
Rinas,  Albania.6 

On  the  other  hand,  a  distributed  system  also  has  its  problems.  The 
widely  varying  and  incomplete  resources  at  some  of  the  sites  clearly 
might  mean  that  the  quality  of  the  fusion  process  could  be  uneven. 
For  example,  it  is  likely  that  not  enough  imagery  analysts  will  be 
available  at  enough  sites  within  a  theater  to  support  local  comman¬ 
ders  adequately.  In  addition,  it  is  likely  that  there  will  not  be  enough 
bandwidth  available  to  allow  all  sites  to  support  local  fusion.  The 
demands  on  the  communications  networks  in  either  case  are  con¬ 
siderable. 

Completeness 

The  measure  of  completeness  for  the  network  subdomain  is  essen¬ 
tially  a  network  reliability  measure  applied  to  the  dissemination  of 
the  CROP  to  all  its  users.  Not  all  users  require  the  complete  CROP, 
and  not  all  require  the  same  level  of  resolution.  However,  the  users 
must  all  share  the  same  information.  Completeness  therefore  mea¬ 
sures  how  well  the  communications  network  accommodates  the 
transmission  of  relevant  aspects  of  the  CROP  to  each  user.  It 
addresses  the  likelihood  that  each  user  will  receive  the  relevant  por¬ 
tions  of  the  CROP.7  A  possible  metric  for  this  measure  is  the  prob¬ 
ability  that  all  users  will  receive  the  CROP.  This  is  an  assessment  of 
the  network’s  reliability  in  terms  of  its  robustness. 

Several  factors  affect  the  reliability  of  a  communications  network, 
foremost  among  them  the  degree  to  which  it  provides  alternative 
transmission  routes.  Other  factors  include  the  degree  to  which  facili¬ 
ties  are  hardened,  enemy  jamming  and  other  electronic  warfare 
attacks,  and  bandwidth.  To  develop  a  metric  for  overall  network  reli¬ 
ability,  it  is  necessary  to  (1)  assess  the  probability  that  each  user  can 


6This  and  other  incidents  concerning  the  deployment  of  Task  Force  Hawk  during 
Operation  Allied  Force  is  documented  in  Nardulli  et  al.  (2002). 

7The  subsection  below  on  correctness  discusses  the  likelihood  that  it  will  be  "cor¬ 
rectly”  received  and  that  the  same  information  is  received. 


46  Exploring  Information  Superiority 


receive  a  message,  then  (2)  combine  these  to  assess  the  probability 
that  all  users  can  receive  a  message.  The  first  step  requires  the 
assessment  of  link  reliabilities  for  all  paths  from  sender  to  receiver. 
The  second  implies  the  assessment  of  a  joint  probability. 

Suppose  we  let  p,  represent  the  probability  that  user  i  will  receive  the 
CROP  from  the  central  fusion  facility,  based  on  all  the  possible  routes 
it  could  take.  The  overall  network  reliability  therefore  would  be 

N-uii  Pi’ 

where  k  is  the  total  number  of  users.  Implicit  in  this  product  is  an 
independence  assumption:  that  the  probability  that  user  i  receives 
the  message  is  independent  of  the  probability  that  user  j  receives  it.8 

The  completeness  component  of  the  transformation  function  de¬ 
picted  in  Figure  3.1  would  therefore  be  QCOm(Ib|Fo>Fi,F2)  =  N. 

Correctness 

Network  correctness  can  be  measured  in  terms  of  the  likelihood  that 
all  users  receive  the  same  CROP  or  portion  of  the  CROP  that  was 
transmitted  to  each  from  the  central  fusion  facility.  This  is  an 
assessment  of  the  likelihood  that  CROP  users  receive  the  distributed 
information  without  degradation. 

One  way  to  measure  this  likelihood  is  to  use  the  probability  of  correct 
message  receipt  (PCMR)  as  a  metric.  As  with  completeness,  using 
PCMR  as  a  metric  requires  a  two-step  assessment.  First,  we  deter¬ 
mine  the  PCMR  for  each  user,  then  expand  it  to  include  all  users.  The 
PCMR  is  essentially  a  conditional  probability  that  the  message  sent 
will  be  the  message  received.  Applied  to  the  dissemination  of  the 
CROP,  the  probability  that  user  i  will  receive  the  CROP  (or  portion  of 
it)  as  transmitted  from  the  central  fusion  facility  is  i3,(F3|F2) .  As  with 
the  completeness  metric,  these  probabilities  are  related  to  the  quality 
of  the  channel(s)  over  which  the  information  is  transmitted.  There¬ 
fore,  such  things  as  signal-to-noise  ratio  (SNR),  bandwidth,  and 
jamming  are  also  determinants  of  the  PCMR.  However,  the  value  is 


8Appendix  B  develops  the  reliability  estimates  for  the  links  in  more  detail. 
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also  dependent  on  the  input  to  the  channels.  The  PCMR,  then,  is  the 
joint  probability  that  the  central  fusion  facility  transmitted  the  CROP 
correctly  and  that  the  user  received  the  observed  CROP  correctly: 
P((F3,F2)=P(F2)Pi.(F3|F2).9 

The  problem,  then,  is  to  find  adequate  representations  for  the 
marginal  probability,  P(F2),  and  the  conditional  probabilities, 
P,(F3|F2).  The  second  of  these  is  taken  to  be  dependent  on  the  relia¬ 
bility  of  the  communications  paths  between  the  central  fusion  facil¬ 
ity  and  the  user.  This  is  the  connectivity  probability,  pit  discussed 
above,  under  completeness.  The  marginal  probability,  P(F2),  is  the 
probability  that  the  CROP,  or  portion  of  it,  will  be  transmitted  cor¬ 
rectly  and  is  therefore  a  function  of  the  communications  equipment 
and  personnel  within  the  fusion  facility.  Given  that  these  probabili¬ 
ties  can  be  assessed,  the  PCMR  for  user  i  is  PCMR,:  =  P,(I^,F2)  = 

The  overall  network  PCMR  can  be  thought  of  as  the  joint  probability 
of  all  the  fusion  facility-to-user  PCMRs: 

PCMR  =  nt,P(F2)p(-. 

The  correctness  component  of  the  transformation  function  depicted 
in  Figure  3.1  would  therefore  be  Qcor(F3|F0,F1,F2)  =  PCMR. 

Currency 

Network  currency  depends  on  the  rate  at  which  data  can  be  trans¬ 
mitted  over  the  network’s  links  and  the  amount  of  time  required  to 
process  information  at  the  various  nodes  in  the  network.  The  former 
depends  on  bandwidth  and  the  complexity  of  the  communications 
paths  between  subscribers,  and  the  latter  depends  on  the  activities 
performed  at  the  nodes.  The  end-to-end  time  required  to  transmit 
the  CROP  from  the  central  fusion  facility  to  the  users  can  therefore 
serve  as  a  measure  of  network  currency. 


9See,  for  example,  Blahut  (1988)  for  further  discussions  of  probability  distributions  for 
sources  and  channels. 
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Calculating  these  times  can  be  formidable.  Depending  on  the  degree 
of  connectivity  in  a  network,  the  number  of  ways  to  get  a  message 
between  two  nodes  can  be  considerable.  Given  the  myriad  ways  to 
get  a  message  between  two  nodes,  how  should  we  assess  the  end-to- 
end  transmission  time?  There  are  several  possibilities.  A  conservative 
approach  is  to  calculate  the  time  required  along  the  “longest”  path. 
Another  is  to  select  the  minimum,  and  still  another  might  be  to 
calculate  the  average  of  all  paths.  For  this  analysis,  we  chose  the 
average  transmission  and  processing  times.  The  overall  average 
network  transmission  delay,  then,  is  the  average  of  these  times,  t ,  so 
that  the  currency  component  of  the  transformation  function  in 
Figure  3.1  is 


( ^3 1 F0 ,  Fi ,  F2 )  —  e  f 


An  Alternative  for  Complete  Networks 

The  methodology  we  have  suggested  for  calculating  completeness, 
correctness,  and  currency  of  a  communications  network  becomes 
rather  tedious  when  the  size  and  connectivity  of  the  network 
increase.  In  the  limit,  for  example,  every  node  is  a  relay,  and  every 
entity  on  the  battlefield  is  a  node.  The  Army  refers  to  this  as  a 
“nodeless  network,”  the  idea  being  that,  with  such  a  richly  connected 
network,  the  loss  of  a  node  will  not  degrade  performance  in  any 
appreciable  way.10 

The  fully  connected,  or  complete,  network  is  a  subset  of  the  nodeless 
network.  A  complete  network  of  n  nodes,  in  which  all  connections 
are  two-way,  has  n{n- 1)  -n2  -  n  connections.  Since  we  are  interested 
only  in  alternative  paths  from  one  node  to  another,  a  large  n  is  desir¬ 
able.  Consequently,  given  that  we  have  a  complete  network,  a  surro¬ 
gate  for  network  quality  in  all  of  its  dimensions  might  be  the  size  of 
the  network.  For  a  very  large  n,  the  first  term,  n2,  dominates,  and  the 
quality  of  the  network  therefore  increases  as  the  square  of  the  num¬ 
ber  of  nodes  in  the  network.  This  is  referred  to  as  Metcalf’s  Law, 


10Taken  from  a  briefing  presented  by  the  U.S.  Army  Communications-Electronics 
Command  Research  Development  and  Engineering  Center,  Fort  Monmouth,  New 
Jersey  (Nichols,  1999). 
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named  for  Robert  Metcalf,  a  pioneer  in  the  development  of  the  Eth¬ 
ernet.11  The  transformation  function  for  currency  can  then  be  writ¬ 
ten  as 


Qcur(p3  I  ^0 ' Ft , F2 )  -  l-e  . 

In  addition  to  mapping  the  size  of  the  network  to  a  0-1  value,  the 
exponential  form  of  the  function  rapidly  approaches  1  initially,  then 
levels  off  as  the  size  of  the  network  increases — a  reasonable  assump¬ 
tion,  since,  after  a  certain  size,  node  losses  are  unlikely  to  degrade 
the  network  significantly. 

Shared  Information 

The  CROP  produced  at  the  central  fusion  facility  is  distributed  to  the 
users  over  the  dissemination  network;  that  is,  it  is  shared  among  the 
users.  Shared  information  is  essential  for  effective  collaboration,  as 
we  will  discuss  more  fully  in  Chapter  Four.  However,  because  effec¬ 
tively  sharing  information  depends  so  much  on  the  structure  of 
communications  networks,  it  seems  appropriate  to  elaborate  a  bit 
here. 

Earlier,  we  referred  to  the  CROP  the  users  receive  as  the  observed 
CROP.  Recall  that  matrix  F2  represents  the  fused  CROP,  which  con¬ 
tains  estimates  of  the  features  or  characteristics  of  the  enemy  units 
detected.  Not  all  friendly  users  need  all  the  information  contained  in 
the  perceived  CROP.  So,  each  user’s  observed  CROP  is  a  subset  of  the 
fused  CROP. 

Figure  3.8  depicts  the  amount  of  information  the  users  receive.  The 
shaded  circles  in  the  diagram  represent  the  amount  of  information 
(the  portion  of  the  CROP)  that  each  of  three  users  receives.  The  val¬ 
ues  on  the  edges  represent  the  probability  that  each  user  will  receive 
the  relevant  portions  of  the  CROP.  The  white  circles  represent  the 
entire  CROP.  The  small  dark  gray  area  in  the  center  of  the  three 


11Metcalf  also  founded  the  3Com  Corporation  of  Santa  Clara,  California,  in  1981,  the 
leading  producer  of  Ethernet  adapter  cards.  Gilder  (1993)  provides  an  interesting  dis¬ 
cussion  of  Metcalf  and  his  “law  of  the  telecosm.”  See  also  Appendix  A  to  Alberts, 
Garstka,  and  Stein  (2000). 


50  Exploring  Information  Superiority 


RAND  MR1467-3.8 


Figure  3.8 — Shared  and  Common  Information 


joined  circles  represents  the  information  that  is  common  among  all 
the  users,  while  the  residual  gray  area  has  the  potential  to  be  shared 
among  all  the  users,  depending  on  the  ability  of  the  group  to  collabo¬ 
rate.  It  is  clear  that  the  ability  to  collaborate  has  the  potential  to 
increase  the  amount  of  information  shared  among  the  users  thus 
contributing  to  shared  situational  awareness,  as  discussed  in  Chapter 
Four. 

SUMMING  UP 

In  this  chapter,  we  applied  the  methodology  we  developed  in  Chap¬ 
ter  Two  to  the  physical  and  information  domains.  We  introduced  the 
feature  matrix,  F  =  [Flt  F&  ,  Fm\,  as  a  set  of  vectors,  Fi}  each  of 
which  represents  the  relevant  features  of  the  enemy.  In  the  physical 
domain,  F0  represents  the  ground  truth  features  of  all  enemy  units. 
The  information  domain  has  three  subdomains:  sensor,  fusion,  and 
network.  This  chapter  assessed  the  three  components  of  information 
quality — completeness,  correctness,  and  currency — for  the  informa¬ 
tion  product  generated  in  each  subdomain. 

The  quantities  developed  here,  in  subsequent  chapters,  and  in 
Appendix  B  are  illustrative  and  do  not  represent  settled  research. 
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Sensor  Metrics 

In  the  sensor  subdomain,  we  examined  three  aspects  of  complete¬ 
ness:  the  number  of  enemy  units  detected,  the  features  reported  for 
the  units  detected,  and  the  sensor  suite’s  coverage  area.  For  infor¬ 
mation  to  be  complete,  all  features  of  all  units  in  the  relevant  ground 
truth  must  be  known,  and  the  entire  AO  must  be  under  sensor  obser¬ 
vation.  The  suggested  completeness  metric  consists  of  two  compo¬ 
nents,  both  of  which  are  between  0  and  1:  cx  is  the  fraction  of  enemy 
units  detected,  and  c2  is  the  fraction  of  the  AO  covered.  The  proposed 
completeness  transformation  metric  then  is 

Qcomffl|Fo)  =  Ci(l-e  c2). 

We  suggested  two  types  of  metrics  for  correctness,  those  designed  to 
support  controlled  experiments  and  those  designed  to  support  oper¬ 
ations,  and  defined  correctness  as  the  degree  to  which  the  true  target 
features  approximate  their  ground  truth  values.  We  suggested  using 
estimation  theory  to  assess  the  deviation  from  ground  truth  for  con¬ 
trolled  experiments.  Since  an  unbiased  estimator  of  a  parameter  is 
one  whose  expected  value  matches  the  true  parameter,  the  differ¬ 
ence  between  the  estimate  and  the  known  ground  truth  appears  to 
be  suitable  for  measuring  correctness.  In  general,  if  A  is  a  measure  of 
nearness,  such  that  near-zero  values  of  A  correspond  to  high  near¬ 
ness,  the  proposed  correctness  transformation  is  Q^lF^Fo)  =  e~A. 

Assessing  correctness  in  support  of  operations  implies  that  ground 
truth  is  not  known.  In  this  case,  we  clustered  the  detections  geo¬ 
graphically  using  a  pattern-classification  technique  and  then  calcu¬ 
lated  the  variance  within  the  cluster.  For  a  location  estimate,  this 
variance  is  expressed  in  terms  of  a  covariance  matrix.  The  determi¬ 
nant  of  that  matrix  is  a  measure  of  precision  and,  therefore,  a  mea¬ 
sure  of  correctness.  The  determinant  is  p  =  S4,  where  S2  is  the  sample 
variance  in  both  the  x  and  y  directions.  The  proposed  correctness 
transformation  then  is  Qcor(Fi|Fo)  =  e~p  ■ 


Fusion  Metrics 

Fusion  is  the  process  of  combining  information  from  sensors  and 
sources  to  produce  a  common,  relevant  picture  of  the  battlespace, 
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the  CROP.  This  process  includes  the  correlation  and  analysis  of  data 
inputs  from  supporting  sensors  and  sources.  Fusion  occurs  at  several 
levels,  from  the  simple  combining  of  tracks  and  identity  estimates  to 
assessments  of  enemy  intent.  Our  focus  here  is  on  the  lower  levels  of 
fusion. 

We  assumed  that  the  architecture  of  the  fusion  suite  consists  of  sev¬ 
eral  fusion  facilities  that  focus  on  a  single  intelligence  discipline. 
These  facilities  transmit  their  fused  reports  to  a  central  facility  that 
produces  a  final  product,  the  CROP. 

Completeness  in  the  fusion  subdomain  focuses  on  the  number  of 
detected  enemy  units  from  the  sensor  subdomain  that  are  classified, 
i.e.,  described  in  terms  of  the  relevant  features.  The  number  the 
fusion  facilities  can  classify  depends  on  the  architecture  of  the  fusion 
suite,  the  degree  of  automation,  and  the  control  procedures  (the  abil¬ 
ity  of  the  system  to  retask  the  sensors).  The  proposed  completeness 
transformation  is 


Qcom(^2  I  F() >  F]  ) 


i-niiM) 


where  c,  is  the  fraction  of  the  detected  enemy  units  fusion  facility  i 
can  process  per  unit  time  and  cc  is  the  fraction  the  central  processing 
facility  can  process. 

Correctness  in  the  fusion  subdomain  measures  how  close  the  fused 
estimate  for  each  enemy  unit  feature  is  to  ground  truth — that  is,  how 
accurate  the  classifications  of  the  reported  detections  are.  One  way 
to  address  this  problem  is  to  examine  the  variance  in  the  feature 
estimates  for  each  reported  unit.  This  results  in  the  following  cor¬ 
rectness  transformation: 


Qcor, i(f2  I  FoFi)  =  Wt  1Phl  CO  j  (es>)  . 

where  wt  and  coj  are  weights,  the  first  accounting  for  the  relative 
importance  of  the  reported  enemy  unit  and  the  second  for  the  rele¬ 
vant  importance  of  the  features  being  reported;  Sj  represents  the 
sample  standard  deviations  for  each  of  the  p  features  for  a  given 
enemy  unit,  derived  from  the  number  of  reports  arriving  on  the  unit; 
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and  the  second  subscript  on  Q  distinguishes  this  correctness 
transformation  from  the  tracking  metric  (discussed  next). 

We  also  chose  to  measure  how  well  we  are  able  to  track  enemy  units. 
The  correctness  of  the  tracks  of  enemy  units  can  be  measured  in 
terms  of  the  number  of  previous  tracks  that  have  been  confirmed  on 
the  present  scan  and  the  number  of  new  tracks  initiated.  We  took  the 
tracking  portion  of  the  correctness  component  of  the  transformation 
function  to  be  QCOr, 2(F2|F0,F1)  =  T,  where  T  is  the  fraction  of  the  enemy 
units  that  correlate  with  previous  tracks. 

Combining  the  two  correctness  metrics  using  an  importance  weight, 
0  <  m  <  1,  we  get  the  following  correctness  component  for  the  trans¬ 
formation  function:  Qcor(F2|Fo>Fi)  =  ®kk+  (1  -  o))T. 

The  metric  for  the  currency  component  of  the  transformation  func¬ 
tion  might  be  QcudJyFo^j)  =  erl,  where  t  is  the  total  time  required  to 
update  the  CROP. 

Network  Metrics 

Completeness  measures  how  well  the  communications  network 
accommodates  the  transmission  of  relevant  aspects  of  the  CROP  to 
each  user.  The  metric  we  chose  for  this  measure  is  the  probability 
that  all  users  will  receive  the  CROP.  This  is  an  assessment  of  the  net¬ 
work’s  reliability  in  terms  of  its  robustness.  The  completeness  com¬ 
ponent  of  the  transformation  function  is 

Qcom  ( F3  |  F0,F1,F2)  =  UllPi  ’ 


where  lc  is  the  number  of  users  of  the  CROP,  and  pt  represents  the 
probability  that  user  i  will  receive  the  CROP. 

Network  correctness  is  an  assessment  of  the  likelihood  that  CROP 
users  will  receive  the  distributed  information  without  degradation. 
One  way  to  measure  this  is  to  use  the  PCMR.  The  PCMR  is  a  condi¬ 
tional  probability  that  the  message  sent  will  be  the  message  received. 
The  probability  that  user  i  will  receive  the  CROP  (or  portion  of  it)  as 
transmitted  is  P,-(F3,F2)  =  P(F2)P((F3|F2),  where  P(F3|F2)  =  pt.  Therefore, 
the  PCMR  for  user  i  is 
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PCMR,-  =  P{  (f3  ,  F2 )  =  p(f2  )p,: 

and  the  correctness  component  of  the  transformation  function  is 
Qcor  (^3  I  Fo  >  Fx ,  F2  )  =  n?=l  PCMRj  . 

The  end-to-end  time  required  to  transmit  the  CROP  from  the  central 
fusion  facility  to  the  users  measures  network  currency.  One 
approach  to  doing  this  is  to  calculate  the  average  of  all  paths  from 
source  to  user.  The  overall  average  network  transmission  delay  is 
then  the  average  of  these  times, 


t , 

so  that  the  currency  component  of  the  transformation  function  is 
Qciir(p3  I  Fo’Fi’^)  =  e  ■ 


Shared  Information 

Shared  information  is  an  essential  ingredient  of  effective  collabora¬ 
tion,  as  Chapter  Four  will  discuss  more  fully.  We  refer  to  the  CROP 
received  by  the  users  as  the  observed  CROP.  Matrix  F2  represents  the 
fused  CROP.  Each  user’s  observed  CROP  is  a  subset  of  the  fused 
CROP.  The  overlap  among  these  subsets  constitutes  the  information 
shared  among  the  users.  Information  not  in  the  overlap  has  the 
potential  to  be  shared  through  the  process  of  collaboration.  The 
ability  to  collaborate  therefore  has  the  potential  to  increase  the 
amount  of  information  shared  among  the  users,  thus  contributing  to 
shared  situational  awareness. 


Chapter  Four 

THE  COGNITIVE  DOMAIN 


The  more  we  accurately  search  into  the  human  mind,  the 
stronger  traces  we  everywhere  find  of  the  wisdom  of  Him 
who  made  it. 


— Edmund  Burke 


In  the  information  domain,  the  data  collected  on  the  physical 
domain  are  processed  and  disseminated  to  friendly  users.  The  prod¬ 
uct  produced  is  the  observed  CROP,  consisting  of  a  function  of  the 
set  of  feature  vectors,  F0,  F1(  F2,  and  F3.  The  quality  of  the  information 
used  to  produce  the  picture  depends  on  the  functional  architecture 
in  each  of  the  information  subdomains.  In  the  cognitive  domain,  the 
products  of  the  information  domain  are  used  to  take  decisions.  The 
mental  processes  that  transform  the  CROP  into  a  decision  and  a  sub¬ 
sequent  action  are  not  well  understood,  beyond  the  fact  that  they 
follow  a  general  progression  from  awareness  (a  person’s  cognitive, 
holistic  view  of  the  battlespace)  to  understanding  (the  extraction  of 
meaning  from  the  holistic  battlespace  view)  to  decisionmaking. 
These  processes  depend  on  a  range  of  factors,  a  few  of  which  are 
psychological. 

Here  we  focus  on  awareness  and  shared  awareness,  the  steps  before 
understanding,  decision,  and  subsequent  action  take  place.  With 
respect  to  the  NCW  Conceptual  Framework,  we  are  focusing  on  the 
awareness  attribute  subsets  of  the  Quality  of  Individual  Sense 
Making  and  Degree  of  Shared  Sensemaking  measures.  The  cognitive 
processes  that  transform  the  CROP  into  a  decision  and  subsequent 
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action  must  be  described  for  participants  in  the  decision  process 
both  as  individuals  and  as  interacting,  collaborating  members  of  a 
decisionmaking  team.  Continuing  the  Information  Superiority  Value 
Chain  depicted  in  Figure  2.2  through  to  shared  situational  aware¬ 
ness,  we  get  the  transformation  diagram  depicted  in  Figure  4.1. 

Our  objective  is  to  describe  a  modeling  framework  that  begins  to 
reflect  the  factors  most  likely  to  influence  individual  situational 
awareness,  shared  situational  awareness,  and  collaboration  prior  to 
decisionmaking.  This  chapter  emphasizes  completeness  measures 
and  metrics.  Our  approach  is  to  establish  a  theoretical  foundation  by 
developing  definitions,  metrics,  and  hypotheses  regarding  their  rela¬ 
tionships. 

ANALYSIS  IN  THE  COGNITIVE  DOMAIN 

The  decisionmaker  must  possess  some  degree  of  situational  aware¬ 
ness  to  understand,  i.e.,  draw  inferences  from  the  picture  presented 
to  him.  Regardless  of  how  complete,  correct,  and  timely  the  CROP 
presented  to  the  commander,  there  is  no  guarantee  that  he  will  be 
cognizant  of  the  relevant  knowledge  reflected  in  the  CROP.1  An 
important  question,  then,  is  how  correct,  complete,  and  current  the 
information  must  be  to  ensure  sufficient  understanding  for  the  vari¬ 
ous  levels  of  awareness  and,  ultimately,  an  accurate  assessment  of 
the  situation  the  commander  is  facing.  The  answer  depends  on  a 
number  of  factors  including  the  decisionmaker,  the  decision  support 
system,  and  the  collaborative  environment.  Consequently,  it  is 
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Figure  4.1 — Cognitive  Domain  Transformations 


1See  Appendix  A  for  definitions  of  the  italicized  terms. 
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important  to  develop  a  relationship  between  the  various  quality 
metrics  and  situational  awareness  as  a  function  of  the  key  parame¬ 
ters  that  characterize  these  ingredients. 

Realizing  the  complexity  of  representing  the  functional  relationships 
between  the  various  quality  metrics  and  awareness,  we  chose  to 
address  them  in  the  following  order: 

•  Situational  awareness  of  the  individual.  First,  model  an  individ¬ 
ual  decisionmaker’s  situational  awareness,  given  key  attributes 
characterizing  individuals  and  information  quality  from  the 
Information  Domain; 

•  The  collaboration  process  for  a  team.  Next,  model  the  role  of 
information  available  to  individuals  participating  together  in 
some  joint  collaborative  action;  and 

•  Shared  situational  awareness  of  the  team.  Finally,  modify  the 
model  of  an  individual’s  situational  awareness  to  include  factors 
representing  information  available  to  collaborating  individuals, 
then  extend  this  model  to  the  situational  awareness  of  an  entire 
collaborating  team. 


MODELING  INDIVIDUAL  SITUATIONAL  AWARENESS 

Identifying  what  it  will  take — the  level  of  information  quality — for  a 
decisionmaker  to  assess  the  situation  presented  to  him  correctly — to 
“be  aware” — is  complex.  Several  factors  come  into  play:  education 
and  training,  experience,  the  current  situation,  cultural  background, 
personality,  language,  the  opportunity  to  collaborate  with  others,  the 
quality  of  the  information  presented,  etc.  We  propose  the  following 
metric  for  individual  situational  awareness: 

Degree  of  Situational  Awareness:  the  fraction  of  fused  feature  vec¬ 
tors  the  decisionmaker  is  capable  of  realizing. 

Stated  this  way,  this  equates  to  the  degree  to  which  the  decision¬ 
maker  is  aware  of  the  situation  facing  him.  This  metric  emphasizes 
the  use  of  the  individual  components  of  the  CROP  and  includes  a 
reference  to  the  ability  of  the  individual  decisionmaker.  It  does  not 
place  greater  or  lesser  value  on  the  decisionmaker’s  correct  realiza¬ 
tion  of  fused  feature  vectors;  rather,  it  focuses  exclusively  on  his  abil- 
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ity  to  realize  what  he  sees.  It  is  essentially  a  completeness  metric. 
Additional  metrics  (e.g.,  correctness  of  awareness)  remain  to  be 
developed  in  future  research. 

Modeling  the  Individual 

It  is  impossible  to  deal  with  all  the  factors  that  contribute  to  an  indi¬ 
vidual  decisionmaker’s  level  of  situational  awareness  in  any  coherent 
way.  Instead,  we  chose  to  use  an  agent  representation  of  a  decision¬ 
maker.  That  is,  some  combination  of  the  factors  listed  in  the  previous 
paragraph  will  predispose  the  commander  to  grasp  the  situation 
quickly,  and  some  combinations  will  not.  In  the  absence  of  research 
in  this  area,  we  simply  posited  a  continuum  and  selected  discrete 
points  on  that  continuum.  For  example,  suppose  we  focus  initially 
on  education  and  training,  experience,  and  familiarity  with  the  cur¬ 
rent  situation.  For  all  these  factors,  the  domain  is  clearly  continuous. 
Rather  than  deal  with  the  complexities  of  continuous  domains,  we 
instead  defined  two  discrete  points  for  each,  as  shown  in  Table  4.1. 

A  strict  combinatorial  assessment  of  the  example  attributes  in  Table 
4.1  produces  eight  distinct  possible  awareness  conditions  that  char¬ 
acterize  a  commander’s  predisposition  to  grasp  the  situation  pre¬ 
sented  to  him.  We  refer  to  these  as  decision  agents.  Suppose  we  focus 
on  just  four  that  descend  in  the  order  of  awareness  of  the  situation 
presented.  These  are  described  in  Table  4.2,  where  the  exemplar 
decision  agents  are  denoted  by  4>,. 

Next,  we  interpreted  A  =  h[Q(F0,F1,F2,F3),F4  ]  E  [0,1]  to  be  the  degree 
(level)  of  individual  situational  awareness  each  of  the  commander 
decision-agent  types  possesses.  Recall  that  Q(F0,F1,F2,F3)  is  the  qual¬ 
ity  (completeness  in  this  case)  of  the  CROP  produced  in  the  informa¬ 
tion  domain,  and  F4  is  the  observed  CROP.  For  example,  a  comman¬ 
der  who  is  totally  aware  of  the  situation  presented  to  him — one  who 
is  able  to  realize  almost  all  the  observed  CROP — has  a  score  close 
to  l.2  The  remaining  question  is  how  the  quality  of  the  information 
represented  in  the  CROP  influences  the  commander’s  awareness.  It 


2Note  that  total  situational  awareness  does  not  equate  to  “willingness  to  decide.”  It 
may  be  that  the  commander  is  “aware”  of  the  fact  that  he  has  insufficient  information 
to  act  wisely — that  is,  insufficient  information  to  make  appropriate  inferences  about 
the  situation. 
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seems  reasonable  to  assume  that,  regardless  of  the  commander’s 
inherent  awareness  characterization,  information  of  higher  quality 
will  tend  to  increase  awareness  over  some  range.3  Consequently,  we 

Table  4.1 

Exemplar  Discrete  Awareness  Attributes 


Attribute 

High 

Low 

Education  and 
training 

Graduate  of  advanced 
service  and  civilian 
schools 

Limited  education  beyond 
undergraduate  studies 
and  basic  service  school 

Experience 

Senior  officer  who  has 
commanded  troops 
in  several  operations 

lunior  officer  with  limited 
combat  experience 

Current  situation 

Familiar  with  the  cur¬ 
rent  situation 

Unfamiliar  with  the  current 
situation 

NOTE:  The  entries  in  this  table  and  in  Table  4.2  are  notional  and  are  not  the 
result  of  settled  research.  Clearly,  several  other  attributes  and  combinations  of 
attributes  may  be  at  play. 


Table  4.2 

Exemplar  Decision  Agents 


Agent 

Characterization 

Description 

®l 

Highly  capable 

An  experienced,  well-educated  commander  who  is 
familiar  with  the  situation  confronting  him  and  a 
veteran  of  considerable  field  training 

<|>2 

Less  capable 

An  experienced  commander  with  limited  educa¬ 
tion  who  is  unfamiliar  with  the  situation  con¬ 
fronting  him  but  has  some  field  training 

d>3 

Marginally 

capable 

An  inexperienced  commander  with  limited  educa¬ 
tion  who  is  unfamiliar  with  the  situation  con¬ 
fronting  him  but  has  some  field  training 

®4 

Incapable 

An  inexperienced  commander  with  limited  educa¬ 
tion  who  is  unfamiliar  with  the  situation  con¬ 
fronting  him  and  has  little  field  training 

3Clearly,  this  is  another  area  in  which  additional  research  is  needed. 
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sought  a  functional  relationship  in  which  the  dependent  variable  is 
“degree  of  awareness,”  A,  and  the  independent  variables  are  infor¬ 
mation  quality  measures  (completeness,  in  this  case). 

Individual  Situational  Awareness 

The  remaining  chore  is  to  relate  the  awareness  of  the  alternative 
decision  agents  to  each  of  the  total  information  quality  measures. 
This  suggests  isorelationships  for  each  of  the  measures.  Recall  that 
the  situational  awareness  range  (the  dependent  variable)  is  A  £  [0,1]. 
For  completeness  and  correctness  of  situational  awareness,  the 
domain  can  also  be  between  0  and  1.  What  is  needed  is  a  relationship 
that  shows  decision  agents  with  high  awareness  (<b,  and  4>2)  becom¬ 
ing  markedly  more  aware  of  the  situation  with  increasing  complete¬ 
ness  and  correctness  of  information,  and  less  so  for  decision  agents 
with  low  awareness  (fl>3  and  <b4).  Figure  4.2  is  a  notional  illustration  of 
such  a  relationship  for  completeness. 

The  four  curves  are  isofunctions  whose  shape  depends  on  the  ability 
of  the  individual  decision  agent  as  described  by  the  parameter,  <&,. 
Consequently  we  modify  the  transformation  depicted  in  Figure  4.1 
so  that  we  have  A9i-  h[Q(F0,F1,F2,F3),F4].  The  subscripted  notation 
will  be  used  from  here  on  through  the  rest  of  the  report.  Figure  4.2 
therefore  depicts  the  degree  of  situational  awareness  that  the  four 
notional  decision-agent  capability  levels  attain  as  the  quality 
(completeness)  of  the  CROP  varies. 

The  Effects  of  Fusion  Levels 

The  level  of  situational  awareness  of  an  individual  will  likely  be 
affected  by  the  level  of  fusion  in  the  information  processing  system 
supporting  the  decisionmaking  process  (e.g.,  the  type  of  display  and 
the  format  for  presentation  of  the  information) .  The  levels  of  fusion 
were  defined  in  Table  3.1.  For  example,  the  situational  awareness 
curves  for  the  same  four  decisionmakers  depicted  in  Figure  4.2  might 
resemble  those  depicted  in  Figure  4.3  at  a  lower  level  of  fusion. 
Suppose  that  Figure  4.2  represented  awareness  at  fusion  level  3,  i.e., 
the  fused  CROP  contains  “assessments  of  enemy  intent.”  Figure  4.3 
might  then  represent  awareness  at  fusion  level  1,  a  simple  set  of 
“refined  tracks  and  identity  estimates.” 
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Figure  4.2 — Notional  Effect  of  Complete  Information  on  Awareness 


Figure  4.3 — Fusion- Level  Effects  on  Situational  Awareness 
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MODELING  SHARED  SITUATIONAL  AWARENESS 

Individual  awareness  must  now  be  refined  to  account  for  the  effects 
of  individuals  participating  in  some  joint  action  prior  to  decision¬ 
making.  That  is,  we  wish  to  explicitly  represent  the  effects  of  shared 
situational  awareness  on  the  individual  decisionmaker  as  the  first 
step  toward  measuring  the  overall  effects  of  shared  situational 
awareness  for  a  collective  decision  team.  Our  model  of  individual 
situational  awareness  thus  far  includes  a  representation  of  the  differ¬ 
ences  among  individual  decision  agents  and  the  quality  of  informa¬ 
tion  produced  in  the  information  domain.  To  describe  shared 
situational  awareness,  we  must  augment  the  current  model  by 
representing  the  complex  interactions  in  situations  involving  more 
than  one  individual.  We  propose  the  following  metric: 

Degree  of  Shared  Situational  Awareness:  the  fraction  of  fused  fea¬ 
ture  vectors  in  the  CROP  that  members  of  a  team  realize  similarly, 
whether  or  not  they  collaborate. 

This  metric  emphasizes  the  importance  of  individual  situational 
awareness  and  allows  agreement  to  exist  even  when  individual  deci¬ 
sionmakers  have  not  collaborated.  Collaboration  may  not  be  needed 
when  (1)  information  quality  is  very  good,  (2)  each  of  the  comman¬ 
der  agents  is  highly  capable  and  knows  that  the  other  commander 
agents  are  similarly  capable,  and  (3)  the  situation  at  hand  is  not 
unusual  or  complex.  Consider,  for  example,  an  experiment  in  which 
information  is  sent  to  many  people.  Each  person  individually  inter¬ 
prets  that  information  received.  People  are  then  interviewed  to 
ascertain  what  they  thought  they  knew  about  the  situation.  An  out¬ 
come  of  this  experiment  could  be  that  these  persons  have  shared 
perceptions  even  though  they  did  not  collaborate.  Their  beliefs  about 
what  other  persons  thought  or  perceived  may  not  have  mattered. 

The  metric  also  considers  the  effects  of  collaboration  during  joint 
action.  Joint  actions  are  sequences  of  jointly  performed  acts  occur¬ 
ring  within  some  process  over  a  period  of  time.  A  joint  act  is  a  single 
instance  of  a  coordination  action  undertaken  by  two  or  more  indi¬ 
viduals.  loint  actions  require  the  coordination  of  what  the  partici¬ 
pants  intend  to  do  and  the  physical  and  mental  states  required  to 
execute  those  intentions  (see  Clark,  1996,  p.  59).  The  metric  we 
developed  for  degree  of  shared  situational  awareness  emphasizes  the 
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importance  of  factors  affecting  individual  situational  awareness,  the 
quality  of  collaboration,  the  influence  of  collaboration  on  individual 
decision  agents  (including  the  importance  of  belief  about  presup¬ 
positions,  as  this  chapter  discusses  later),  and  factors  representing 
situation  complexity.  The  focus  here  will  be  on  modeling  the  quality 
of  collaboration  and  the  influence  of  collaboration  on  individual 
decision  agents. 

We  approached  this  modeling  by  formulating  hypotheses  about 
these  complex  interactions  and  their  effects  on  shared  situational 
awareness.  We  have  not  attempted  to  test  these  hypotheses  in  any 
rigorous  way;  rather,  we  assumed  their  validity  and  built  models  that 
are  consistent  with  them.  The  exemplar  measures  and  metric  sug¬ 
gested  in  Appendix  B  follow  from  the  models.  Our  research  suggests 
that  the  hypotheses  presented  in  the  subsequent  subsections  collec¬ 
tively  define  a  reasonable  theoretical  starting  point  for  modeling 
shared  situational  awareness. 

Collaborating  Teams 

We  first  hypothesized  that,  when  employed,  collaboration  is  critical 
for  determining  shared  situational  awareness.  As  mentioned  earlier, 
under  some  circumstances,  shared  situational  awareness  could  be 
achieved  without  collaboration.  In  addition,  the  effects  of  collabora¬ 
tion  can  be  positive  or  negative.  They  are  positive  when  they  fill  gaps 
in  knowledge  or  confirm  existing  knowledge  and  are  negative  when 
gaps  remain  and  evidence  received  disconfirms  what  is  currently 
“known.”  In  the  latter  case,  time  is  required  both  to  gather  more 
information  and  to  resolve  inconsistencies.  For  now,  however,  we 
focus  on  assessing  the  important  attributes  that  affect  teams  that  do 
collaborate  and  therefore  have  either  positive  or  negative  impacts  on 
the  degree  of  shared  awareness. 

Two  significant  categories  of  attributes  affect  collaborating  teams: 
individual  and  group.  Table  4.3  lists  some  of  the  attributes  in  each 
category.  Both  individual  and  group  attributes  are  important  for 
describing  collaboration;  therefore,  we  considered  both  in  modeling 
shared  situational  awareness.  Of  concern,  however,  is  identifying  the 
factors  that  affect  interactions  among  collaborators  and  selecting 
those  that  have  the  most  effect.  The  literature  seems  to  suggest  that 
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Table  4.3 

Attributes  Affecting  Collaborating  Teams 


Characteristics 

Attributes 

Individual 

Experience 

Familiarity  with  situations  similar  to  the 
current  one 

Ability  to  share  knowledge 

Ability  to  access  the  knowledge  of  others 
Access  rights 

Authority  level 

Competence  with  the  collaboration  tools 

Group 

Task  structure 

Role  specification 

Shared  operational  model 

Degree  of  common  language 

Group  dynamics 

Quality  of  the  interoperability  the  collabo¬ 
ration  environment  provides 

formality  of  interaction,  group  size,  group  roles,  and  task  complexity 
have  the  greatest  effect.4  We  derived  our  structural  model  for  collab¬ 
orative  team  interaction  from  these  attributes  and  factors. 

Common  Ground 

Herbert  Clark  (1996,  p.  12)  has  defined  common  ground  as  “the  great 
mass  of  knowledge,  beliefs,  and  suppositions  [participants]  believe 
they  share.”  Robert  Stalnaker  (1978,  p.  321)  described  common 
ground  in  terms  of  what  he  called  “presuppositions”: 

Roughly  speaking,  the  presuppositions  of  a  speaker  are  the  proposi¬ 
tions  whose  truth  he  takes  for  granted  as  part  of  the  background  of 
the  conversation  . . .  Presuppositions  are  what  is  taken  by  the 
speaker  to  be  the  common  ground  of  the  participants  in  the  con¬ 
versation,  what  is  treated  as  their  common  knowledge  or  mutual 
knowledge,  [emphasis  added] 


4Two  others  mentioned  are:  time  and  task  dependency,  and  the  physical  location  of 
collaborating  team  members. 
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Both  definitions  imply  that  some  sort  of  joint  activity  is  taking  place 
and  that  common  ground  describes  what  the  participants  believe 
they  share.  In  this  context,  “[A]  joint  action  is  one  that  is  carried  out 
by  an  ensemble  of  people  acting  in  coordination  with  each  other” 
(Clark,  1996,  p.  3).  Joint  activities  can  be  represented  by  the  sequence 
of  states  attained  by  the  joint  activity  and  the  trace  of  moves  or 
actions  that  caused  the  state  changes.  During  a  joint  activity,  com¬ 
mon  ground  accumulates  among  participants. 

We  adopted  a  version  of  Clark’s  definition  (see  Appendix  A).  This 
suggests  that,  at  any  moment  in  a  joint  collaborative  process,  com¬ 
mon  ground  consists  of  the  three  components  listed  in  Table  4.4, 
along  with  the  corresponding  information  attributes. 

Collaboration  and  Common  Ground 

A  second  hypothesis  emphasizes  the  importance  of  common  ground 
to  the  collaboration  process.  To  be  effective,  collaboration  requires 
both  the  development  of  common  ground  among  collaborators  and 
familiarity  with  the  capabilities  of  other  collaborators.  We  consider 
two  varieties  of  common  ground:  actual  and  assumed.  Because 


Table  4.4 

Common  Ground  Components  and  Attributes 


Component 

Definition 

Information  Attributes 

Initial  common 
ground 

The  background  facts, 
assumptions,  and  beliefs 
participants  presuppose 
at  the  start  of  a  joint 
activity 

Information  that  is  offi¬ 
cially  part  of  the  joint 
activity 

Current  state  of 
the  joint 
activity 

The  presuppositions  each 
participant  has  about  the 
current  state  of  the 
activity 

Information  that  is  not 
officially  part  of  the 
joint  activity 

Public  events  so 
far 

The  events  participants 
presuppose  have 
occurred  in  public 
leading  up  to  the  current 
state 
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common  ground  is  defined  by  what  people  think  they  share,  there 
may  be  differences  between  what  they  think  they  shared  and  what 
they  actually  share.  The  transition  process  from  assumed  common 
ground  to  actual  common  ground  may  not  be  uniform  for  all  group 
knowledge,  beliefs,  and  suppositions. 

Common  ground  does  not  develop  instantaneously  when  there  is 
collaboration.  Clark  approached  the  development  of  common 
ground  through  the  passage  of  states  in  joint  actions.  Morris  Friedell 
(1967,  pp.  28-39)  introduced  a  Boolean  algebra  model  of  common 
ground.  He  noted  that  there  is  a  period  of  “Initial  Calibration’’  during 
which  participants  “tune  in”  to  each  other  and  move  from  a  state  of 
common  sense  to  states  of  common  opinion  and  common  knowl¬ 
edge.  Friedell  also  based  his  analysis  on  what  each  participant 
believes  about  himself  and  about  others,  defining  belief  as  “a 
proposition  that  a  participant  would  assent  to  if  given  ample  oppor¬ 
tunity  to  reflect”  (Friedell,  1967,  p.  2).  Common  opinion  represents 
perceived  symmetry  of  belief  between  two  or  more  individuals. 
Common  knowledge  is  true  common  opinion  and  is  achieved  when 
“common  knowledge  implication”  exists.  In  this  state,  who  knows 
what  about  subject  x  and  who  knows  what  about  subject  y  are  con¬ 
sidered  to  be  common  knowledge.  This  is  a  critical  change  from 
common  ground,  in  that  it  assigns  specific  responsibility  for  certain 
knowledge  to  specific  individuals  and  states  that  these  responsibili¬ 
ties  are  known  among  all  collaborating  participants. 

These  concepts — “initial  collaboration”  and  “common  knowledge 
implication” — lead  us  to  our  next  step  in  modeling  shared  situational 
awareness:  defining  a  model  for  the  storage  and  retrieval  of  opinion 
and  knowledge  within  a  collaborating  team.  This  model  is  also  the 
vehicle  for  describing  the  transition  from  assumed  to  actual  common 
ground. 

Transactive  Memory  Systems 

FreidelTs  “common  knowledge  implication”  concept  suggests  that  it 
is  important  to  know  who  knows  what  about  a  particular  problem.  A 
positive  outcome  of  collaboration,  if  it  occurs,  may  be  that  shared 
facts  previously  known  only  to  distinct  participants  can  provide  for 
interpretations  that  would  not  have  been  possible  otherwise. 
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The  states  of  a  joint  action  represent  the  succession  of  results  of  joint 
acts.  Traces  represent  the  process  steps  executed  between  two  suc¬ 
cessive  states.  Clark  notes  that  the  current  state  of  an  activity  is  often 
carried  in  an  external  representation,  such  as  a  chessboard.  He  also 
notes  that  discrepancies  develop  between  the  individual  representa¬ 
tions  of  common  ground  that  each  participant  in  a  joint  action 
maintains.  These  discrepancies  are  especially  important  to  the  dis¬ 
cussion  of  coordination  problems — where  two  or  more  people  have 
“common  interests,  or  goals,  and  each  person’s  actions  depend  on 
the  actions  of  the  others”  (Clark,  1996,  p.  62).  The  coordination 
devices  for  such  problems  are  clues  for  coordinating  behavior  or  a 
key  that  is  mutually  recognized  as  such.  Representation  of  these 
clues  and  keys  is  the  next  step.  One  structural  model  for  defining  and 
analyzing  “common  knowledge  implication”  is  Daniel  Wegner’s 
transactive  memory  system,  which  he  defined  to  be  “a  set  of 
individual  memory  systems  in  combination  with  the  communication 
that  takes  place  between  individuals”  (Wegner,  1987,  p.  186).  He  also 
states  that  the  “study  of  transactive  memory  is  concerned  with  the 
prediction  of  group  (and  individual)  behavior  through  an  under¬ 
standing  of  the  manner  in  which  groups  process  and  structure 
information”  (Wegner,  1987,  p.  185).  The  transactive  memory  system 
model  addresses  the  storage  and  movement  of  information  among 
actors,  essentially  recording  the  artifacts  of  the  states  and  traces  that 
define  the  processes  of  joint  actions  in  Clark’s  model  of  common 
ground.  This  suggests  the  following  corollary  to  the  common  ground 
hypothesis: 

The  performance  of  a  collaborative  team  is  strongly  dependent  on 
the  completeness  of  its  system  for  recording  and  retrieving 
information. 

The  transactive  memory  system  appears  to  be  a  reasonable  model  of 
the  primary  factors  affecting  collaboration.  It  is  consistent  both  with 
Clark’s  model  for  common  ground  and  with  cognitive  structures  in 
social  cognition  theory  (Pryor  and  Ostrom,  1987). 

Below,  we  will  first  address  the  structure  and  contents  of  a  transac¬ 
tive  memory  system  for  shared  situational  awareness.  Then  we  pro¬ 
pose  a  process  model  that  captures  the  development  of  the  structure 
and  contents  of  a  transactive  memory  system  for  shared  situational 
awareness. 
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Structure  and  Contents  of  the  Transactive  Memory  System 

An  individual  can  store  information  internally  and  retrieve  it  accord¬ 
ing  to  his  own  encoding,  storage,  and  retrieval  processes.  An  individ¬ 
ual’s  metamemory  includes  knowledge  about  these  processes.  If  an 
individual  stores  information  externally,  the  storage  and  retrieval 
process  must  also  include  the  location  of  the  information.  A  person 
must  remember  whether  a  particular  memory  item  (label)  is  stored 
internally  or  externally  (location)  to  be  able  to  access  the  memory 
item’s  value  (the  item). 

If  externally  stored  information  resides  in  another  person,  a  transac¬ 
tive  memory  system  exists.  Individuals  can  be  assigned  as  informa¬ 
tion  stores  because  of  their  personal  expertise  or  through  circum¬ 
stantial  knowledge  responsibility.  Labeled  items  may  be  uniquely 
assigned  to  participants  in  the  transactive  memory  system  (a  differ¬ 
entiated  system),  or  many  individuals  may  store  subsets  of  the  same 
labeled  items  (an  integrated  system). 

Figure  4.4  depicts  the  components  of  a  transactive  memory  system 
for  shared  situational  awareness.  Each  individual  participating  in  the 
transactive  memory,  Ijt  has  a  set  of  memory  components  (a,  e). 
These  memory  components  capture  the  key  elements  of  the  collabo¬ 
ration.  Clark’s  concepts  of  communal  common  ground — embodying 
shared  expertise  and  including  communal  lexicons,  cultural  facts, 
norms,  and  procedures — is  consistent  with  this  transactive  memory 
model.  His  concepts  of  personal  common  ground — including  per¬ 
ceptual  bases,  personal  diaries,  and  personal  lexicons — are  also  con¬ 
sistent  with  this  transactive  memory  model. 

The  shared  information  CROP  memory  component  represents  the 
portion  of  the  CROP  received  by  individual  Ij.  It  is  possible  that  not 
all  individuals  receive  the  complete  CROP;  therefore,  only  a  subset  of 
the  observed  CROP  may  be  common  to  all  individuals.  The  private- 
role  or  situation-specific  information  memory  component  repre¬ 
sents  the  individual’s  internally  stored  memory  encodings  (labels) 
and  memory  items  (information),  not  shared  with  other  participants 
during  collaboration,  that  affect  the  individual’s  ability  to  interpret 
elements  of  the  CROP.  These  memory  items  may  not  be  shared 
because  of  time  or  other  collaboration  environment  constraints. 
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Figure  4.4 — Transactive  Memory  System  for  Shared 
Situational  Awareness 


The  shared-role  or  situation-specific  information  and  the  external- 
role  or  situation- specific  information  components  are  the  core  of  the 
transactive  memory  system.  These  components  represent  informa¬ 
tion  that  some  individuals  store  externally  in  other  individuals  and 
that  some  individuals  retain  on  behalf  of  other  individuals  in  the 
transactive  memory  system.  There  can  be  direct  links  between  an 
individual  and  the  retrieval  of  a  memory  item  (e.g.,  the  link  between 
Ii  and  I2).  There  can  be  indirect  links  that  take  “hops”  through  the 
transactive  memory  system  until  the  memory  item  is  accessed  (e.g., 
the  link  between  Im  and  /2) .  There  can  also  be  retrieval  failures  in  a 
transactive  memory  system,  for  example,  when  an  individual  to 
whom  external  storage  was  assigned  departs  the  transactive  memory 
system  or  when  an  individual  is  temporarily  incapacitated  so  that  he 
forgets  where  information  is  stored.  The  private  metamemory  com¬ 
ponent  contains  each  individual’s  knowledge  about  the  processes  of 
encoding,  storage,  and  retrieval  for  his  or  her  individual  memory  and 
for  the  external  memory  in  the  transactive  memory  system. 
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The  effects  of  automation  can  be  assessed  using  Figure  4.4.  Compo¬ 
nent  a  of  an  individual’s  transactive  memory  is  affected  by  automa¬ 
tion,  the  effects  of  which  have  been  addressed  in  the  production  of 
the  CROP.  Automation  does  not  affect  components  b  and  e  of  an 
individual’s  transactive  memory.  Automation  can,  however,  affect 
components  c  and  d  of  an  individual’s  transactive  memory.  Compo¬ 
nents  c  and  d  may  be  developed  through  at  least  three  mechanisms: 
nonautomation  means,  automation  support  sharing,  or  previous 
transactive  memory  systems  developed  by  other  individuals  and 
shared  with  the  current  individual. 

Process  Model  for  Development  of  the  Transactive  Memory 
System 

Ulhoi  and  Gattiker  (1999)  defined  an  iterative  process  for  incremen¬ 
tal  knowledge  divergence  and  convergence  in  describing  how  people 
develop  a  conceptual  framework  for  a  technological  problem.  Figure 
4.5  applies  this  iterative  refinement  to  development  of  the  transactive 
memory  system  for  shared  situational  awareness. 

The  key  features  of  this  process  are  the  iterative  stages  of  individual 
information  assessment,  followed  by  team  discussion,  leading  to 
some  state  of  shared  situational  awareness.  The  team  discussion 
period  consists  of  reinforcing  and  refuting  current  beliefs  about  the 
situation.  Knowledge  divergence  results  from  the  presentation  of 
interpretations  from  collaboration  team  members.  Knowledge  con¬ 
vergence  is  the  result  of  consensus  derived  from  assessments  about 
the  beliefs  of  other  team  members  during  the  team’s  discussions.  As 
the  extent  of  prior  collaboration  increases,  team  members  not  only 
pass  through  the  previously  identified  iterative  stages,  they  also 
transition  from  assumed  to  actual  (confirmed)  beliefs  about  what  is 
known  and  shared  by  others  in  the  team.  Assumed  beliefs  dominate 
the  initial  calibration  state.  Confirmed  beliefs  dominate  the  common 
knowledge  state. 

Figure  4.5  depicts  the  three  key  shared  situational  awareness  states 
in  this  process.  In  the  initial  calibration  state,  team  members  have 
achieved  consensus  about  what  roles  each  team  member  can  and 
will  perform  in  the  collaboration,  and  the  team  generates  initial 
alternative  actions  for  further  discussion.  In  the  structured  knowl- 
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Figure  4.5 — Process  Model  for  Developing  the 
Transactive  Memory  System 


edge  state,  team  members  have  begun  to  organize  internally  and 
externally  stored  information  for  the  situation,  and  the  transactive 
memory  is  forming.  The  team  has  progressed  to  consensus  about 
who  knows  what  about  the  situation  and  identifies  alternatives  for 
further  analysis  and  comparison.  In  the  final  state,  common  knowl¬ 
edge,  not  only  do  team  members  reach  consensus  about  who 
believes  what  but  also  about  what  is  true  for  the  situation. 
Alternatives  proposed  are  assessed  against  the  consensus  reached 
about  who  knows  what  and  about  what  is  true. 

No  specific  models  or  metrics  are  associated  with  this  process.  The 
process  is  used  as  a  reference  framework  for  interpretation  of  results 
obtained  from  other  models  proposed  for  shared  situational  aware¬ 
ness. 
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Modeling  Familiarity 

In  his  dissertation  on  KC-135  crew  performance  at  Fairchild  Air 
Force  Base,  Washington,  Daryl  Smith  (1999)  used  the  term  “crew 
hardness”  to  measure  how  often  crew  members  fly  as  an  integral 
team.  We  adopted  the  spirit  of  Smith’s  measure  and  use  the  term 
“team  hardness”  in  our  model  of  shared  situational  awareness. 
Figure  4.6  depicts  the  rotation  of  ten  individuals  in  and  out  of  teams 
during  some  period,  x  . 

As  participants  develop  stronger  relationships  with  other  partici¬ 
pants  through  repeated  or  continued  team  interaction,  the  links 
between  the  participants  become  stronger.  Clark  refers  to  this,  in  the 
language  of  common  ground,  as  the  establishment  of  team  ground¬ 
ing.  From  this,  we  hypothesized  that  the  strength  of  these  links  is 
related  to  the  degree  of  common  ground  between  participants.  This 
suggests  a  second  corollary  to  the  common  ground  hypothesis:  The 
completeness  of  the  system  for  recording  and  retrieving  information 
depends  on  how  frequently  the  team  has  recently  collaborated. 

The  links,  though  intuitively  appealing,  summarize  many  of  the 
attributes  and  interaction  effects  set  out  at  the  start  of  this  discus¬ 
sion.  We  required  a  more  definitive  model  for  strengthening  links 
between  individuals  as  team  hardness  increases,  as  the  links  in 
Figure  4.6  depict.  The  structure  of  the  transactive  memory  system 
then  allowed  us  to  define  a  measure  for  team  hardness. 
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Figure  4.6 — Collaborative  Team  Development 
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Estimating  Team  Hardness 

The  preceding  discussion  provided  an  intuitive  definition  of  team 
hardness:  As  participants  develop  stronger  relationships  with  other 
participants  through  repeated  or  continued  team  interaction,  the 
links  between  the  participants  become  stronger.  The  transactive 
memory  system  contains  explicit  links  capturing  the  amount  of 
shared  information  between  team  members.  Figure  4.4  depicted 
example  links  between  individuals  I\  and  I2  and  between  individuals 
Im,  /3,  and  I2.  A  metric  that  measures  the  complexity  of  those  links, 
the  number  of  “hops”  required  to  retrieve  information,  is  desirable. 
We  developed  estimates  of  link  complexity  by  referring  to  the  labels 
assigned  to  the  transactive  memory  system  components  for  individ¬ 
ual  7s  in  Figure  4.4.  If  the  number  of  externally  stored  items  in  the 
transactive  memory  system  were  measured  and  tracked  as  the  team 
hardens,  we  hypothesize  that 

1.  The  ratio  of  the  number  of  shared-role  or  situation-specific  (c)  and 
external-role  or  situation- specific  information  (d)  to  private-role  or 
situation- specific  information  (b)  elements  would  increase  as  the 
transaction  memory  system  becomes  complete,  where  completeness 
is  relative  to  the  complexity  of  the  situation.  The  resulting 
mathematical  expression  is 


TM  — 

11V1  complex 

2.  The  ratio  of  “hops”  to  the  number  of  shared-role  or  situation- 
specific  (c)  information  elements  in  the  transactive  memory  system 
would  indicate  the  complexity  of  the  transactive  memory  system. 
The  resulting  mathematical  expression  is 
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complex 
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It  is  possible  to  collect  information  indicating  these  ratios  through 
analysis  of  transactions  and  communications  among  individuals 
participating  in  team  exercises,  as  the  team  exercise  progresses.  This 
task  is  left  to  further  research. 
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These  two  hypotheses  served  as  the  basis  for  our  functional  models 
of  team  hardness.  These  models  are  time  dependent.  If  t  represents 
the  time  elapsed  since  the  start  of  the  operation  and  r  represents  the 
length  of  time  the  team  has  been  training  or  operating  together, 
TM[T)  represents  transactive  memory,  where  T-  r  +  f.5  As  with  pre¬ 
vious  completeness  metrics,  the  models  selected  for  transactive 
memory  have  the  property  0  <  TM[T)  <  1. 

With  a  model  for  the  transactive  memory  system  and  a  description  of 
its  completeness,  it  is  next  necessary  to  define  a  process  in  which  the 
transactive  memory  system  is  developed  by  a  collaborating  team.  To 
do  this,  it  is  first  necessary  to  understand  the  role  of  consensus 
among  the  team  members. 

Consensus 

Consensus  plays  a  central  role  in  developing  a  transactive  memory 
system.  It  is  the  majority  opinion  of  a  team  arrived  at  through  active 
collaboration.  Its  definition  implies  the  existence  of  shared  situational 
awareness.  Consensus  was  the  engine  driving  the  knowledge  diver- 
gence-and-convergence  cycles  presented  in  Figure  4.5.  Knowledge 
convergence  is  achieved  by  consensus  about  team  members’  beliefs 
during  the  team’s  discussions.  Understanding  the  nature  of  beliefs 
and  how  they  can  be  represented  in  a  form  that  allows  reasoning 
about  the  interactions  of  persons  participating  in  decisionmaking 
teams  is  therefore  essential  to  our  understanding  of  collaborative 
decisionmaking  and  the  role  of  consensus.  Consensus  is  an  example 
of  a  degree  of  shared  situational  awareness.  Noting  that  not  all  col¬ 
laborating  individuals  have  to  agree  before  a  decision  and  subse¬ 
quent  action  can  take  place,  we  are  interested  in  a  measure  of  the 
degree  of  consensus.  We  hypothesized  that  the  degree  of  consensus 
can  be  estimated  by  the  number  of  pairwise  combinations  of  collabo¬ 
rating  individuals  who  interpret  feature  vectors  similarly .6 

It  is  possible  for  consensus  to  exist  on  a  proposition  that  is  “correct” 
as  well  as  on  one  that  is  “incorrect.”  To  complete  the  foundation  for 
the  modeling  of  shared  situational  awareness,  it  is  necessary  to  frame 


5AppendixB  includes  an  example  of  this. 

6  Appendix  B  includes  an  example  mathematical  statement  of  this  measure. 
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consensus  about  truths  and  about  untruths.  Friedell’s  approach  to 
this  issue  is  to  differentiate  opinion  and  knowledge  (Friedell,  1967). 
Common  opinion  represents  the  common  belief  of  a  team;  common 
knowledge  speaks  to  the  truth  or  untruth  of  that  belief. 

Common  opinion  and  common  knowledge  can  exist  with  or  without 
collaboration.  If  all  commander  agents  expect  all  other  commander 
agents  to  interpret  a  fused  feature  vector  a  particular  way,  common 
opinion  and  common  knowledge  may  develop  without  explicit  col¬ 
laboration.  Our  model  captures  the  distinction  between  “A  knows  x” 
and  “A  believes  x,  and  x  is  true.”  Taken  together,  these  observations 
support  the  hypothesis  that  collaboration  does  not  imply  consensus. 

We  used  the  compositions  of  decisionmakers’  beliefs,  as  represented 
by  their  interpretation  of  fused  feature  vectors  and  defined  by  their 
degree  of  individual  situational  awareness,  Ai>(,  to  model  common 
opinion  in  shared  situational  awareness. 

Shared  Situational  Awareness 

Our  models  of  shared  situational  awareness  integrate  the  modeling 
we  proposed  earlier.  First,  we  placed  the  individual  in  a  team  and 
measured  his  situational  awareness  in  a  team  setting.  Note  that  this 
is  not  the  same  as  team  awareness  but  rather  the  effect  of  team 
dynamics  on  an  individual  member  of  a  collaborative  decisionmak¬ 
ing  process.  The  contribution  is  essentially  derivative  of  the  transac¬ 
tional  memory  function  and,  therefore,  of  team  hardness.  Second, 
we  addressed  the  consensus  that  develops  among  collaborating 
individuals  and  its  affect  on  the  team’s  shared  situational  awareness. 
Finally,  we  accounted  for  diversity  of  decision  agent  capabilities 
among  the  collaborators  to  create  a  composite  model  for  the  degree 
of  shared  situational  awareness. 

Team  participation  can  have  both  salutary  and  deleterious  effects  on 
individual  awareness.  Presumably,  team  participation  produces  pos¬ 
itive  synergies  that  improve  individual  performance.  However,  in 
some  instances,  individual  team  members  with  limited  ability  but 
with  positions  of  authority  might  enforce  their  will  on  the  process,  to 
the  detriment  of  other  individuals  in  the  team.  In  addition,  the  fusion 
subdomain  metrics  indicate  that  quality,  fo[Q(Fo,F1,F2,F3),F4],  may 
decrease  over  time.  This  effect  is  included  in  the  individual  shared 
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awareness  function,  developed  earlier.7  A  revised  measure  of  indi¬ 
vidual  situational  awareness  that  combines  these  factors  and  that 
includes  transactive  memory,  A(bi  (f),  needed  to  be  developed.  The 
revised,  time-dependent  A ( r)  is  the  fraction  of  the  observed  feature 
vectors  the  individual  decisionmaker  with  capability  O,  realizes  with 
the  benefit  of  team  participation,  when  the  operation  has  gone  on  for 
t  time  units.  It  is  therefore  taken  to  be  the  completeness  of  the  CROP 
realized  by  agents  of  capability  4>,  at  time  f.8 

The  next  step  was  to  evaluate  the  situational  awareness  of  the  team 
when  working  together,  SA(f),  recognizing  that  it  too  will  be  time 
dependent.  This  is  what  we  refer  to  as  shared  situational  awareness 
and  is  depicted  without  the  time  dependence  in  Figure  4.1.  In  gen¬ 
eral, 


SA(t)  =  g(Ai.,F5) , 


the  collective  fraction  of  the  observed  CROP,  Q(F0,F1,F2,F3),  that  can 
be  interpreted  by  the  entire  team,  is  our  representation  of  the  degree 
of  consensus.  In  this  formulation,  F5  represents  the  observed  CROP 
realized  by  agents  of  capability  <&,  at  time  t.  This  will  be  a  function  of 
the  individual  situational  awareness  of  the  members  with  different 
capabilities  when  working  in  the  team  environment  and  their  situa¬ 
tional  awareness  of  the  individual  feature  vectors  in  the  observed 
CROP.  For  any  two  team  members,  for  example,  we  wish  to  know 
which  feature  vectors  they  can  jointly  interpret.  This  implies  that  we 
know  not  only  the  fraction  of  the  observed  CROP  that  they  can 
interpret  but  also  which  features  they  can  interpret.  Appendix  B 
develops  an  example  metric  based  on  the  pairwise  assessment  of  the 
interactions  of  team  members. 

SUMMING  UP 

In  the  information  domain,  the  data  collected  on  the  physical 
domain  are  processed  and  disseminated  to  friendly  users.  In  the 
cognitive  domain,  the  products  of  the  information  domain  are  used 


7AppendixB  includes  an  example. 

8Appendix  B  develops  an  example  function  with  the  appropriate  characteristics. 
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to  take  decisions.  The  mental  processes  that  transform  the  CROP 
into  a  decision  and  a  subsequent  action  depend  on  a  range  of  factors, 
a  few  of  which  are  psychological.  This  chapter  focused  on  the  steps 
before  a  decision  is  made  and  subsequent  action  takes  place.  The 
cognitive  processes  that  transform  the  CROP  into  a  decision  and 
subsequent  action  must  be  described  for  participants  in  the  decision 
process,  both  as  individuals  and  as  interacting,  collaborating  mem¬ 
bers  of  a  decisionmaking  team.  We  described  a  modeling  framework 
that  attempts  to  reflect  the  factors  most  likely  to  influence  individual 
situational  awareness,  shared  situational  awareness,  and  collabora¬ 
tion  prior  to  decisionmaking. 

Modeling  Individual  Decisionmaking 

Several  factors  influence  what  it  will  take  for  an  individual  decision¬ 
maker  to  assess  the  situation  presented  to  him  correctly.  Among 
these  is  the  quality  of  the  information  presented.  The  metric  we 
developed,  the  degree  to  which  the  decisionmaker  is  aware  of  the 
situation  facing  him,  emphasizes  the  use  of  the  individual  compo¬ 
nents  of  the  CROP  and  includes  a  reference  to  the  ability  of  the  indi¬ 
vidual  decisionmaker.  It  is  interpreted  to  be  the  fraction  of  the 
observed  CROP  the  decisionmaker  realizes. 

We  suggested  an  agent  representation  of  a  decisionmaker  based  on 
combinations  of  decisionmaker  capability  attributes.  We  defined  two 
discrete  points  for  each  attribute  and  generated  four  decision  agents 
possessing  these  attributes  at  one  of  the  two  points.  Consequently, 
we  suggested  a  functional  relationship  in  which  the  dependent  vari¬ 
able  is  “degree  of  awareness”  and  the  independent  variables  are 
information  quality  measures  (completeness  in  this  case) . 

The  end  result  of  this  process  is  an  explicit  relationship  between  the 
quality  of  the  observed  CROP  and  the  ability  of  the  decisionmaker: 
A*t  =  h[Qi  F0,Fi,F2,F3),F4]. 

Modeling  Shared  Situational  Awareness 

Individual  awareness  was  refined  to  account  for  the  effects  of  indi¬ 
viduals  participating  in  some  joint  action  prior  to  decisionmaking, 
thus  explicitly  representing  the  effects  of  shared  situational  aware¬ 
ness  on  the  individual  decisionmaker.  To  describe  shared  situational 
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awareness,  we  augmented  the  current  model  by  representing  the 
complex  interactions  in  situations  involving  more  than  one  individ¬ 
ual.  The  metric  we  proposed  is  the  fraction  of  fused  feature  vectors  in 
the  observed  CROP  that  members  of  a  team  realize  similarly, 
whether  or  not  they  collaborate.  This  metric  emphasizes  the  impor¬ 
tance  of  individual  situational  awareness  and  allows  agreement  to 
exist  even  though  individual  decisionmakers  have  not  collaborated. 

We  hypothesized,  however,  that  when  employed,  collaboration  is  a 
critical  factor  in  determining  shared  situational  awareness.  We 
focused  on  assessing  the  important  attributes  that  affect  teams  that 
do  collaborate  and  that  therefore  have  either  positive  or  negative 
effects  on  the  degree  of  shared  awareness. 

The  concept  of  common  ground  was  introduced  as  an  ingredient  in 
the  shared  situational  awareness  process.  We  took  it  to  mean  the 
knowledge,  beliefs,  and  suppositions  team  members  believe  they 
share.  During  a  team  activity,  therefore,  common  ground  accumu¬ 
lates  among  team  members. 

We  further  hypothesized  that  to  be  effective,  collaboration  requires 
both  the  development  of  common  ground  among  collaborators  and 
familiarity  with  the  capabilities  of  other  collaborators.  Common 
ground  does  not  develop  instantaneously  when  there  is  collabora¬ 
tion,  there  is  a  period  of  “initial  calibration”  during  which  partici¬ 
pants  “tune  in”  to  each  other  and  move  from  a  state  of  common 
sense  to  states  of  common  opinion  and  common  knowledge. 

A  structural  model  for  defining  and  analyzing  this  phenomenon  is  a 
transactive  memory  system,  defined  as  a  set  of  individual  memory 
systems  in  combination  with  the  communication  that  takes  place 
between  individuals.  This  model  is  concerned  with  the  prediction  of 
group  (and  individual)  behavior  through  an  understanding  of  how 
groups  process  and  structure  information. 

An  individual  can  store  information  internally  and  retrieve  according 
to  his  or  her  own  encoding,  storage,  and  retrieval  processes.  If  an 
individual  stores  information  externally,  the  storage  and  retrieval 
process  must  also  include  the  location  of  the  information.  If  exter¬ 
nally  stored  information  resides  in  another  person,  a  transactive 
memory  system  exists.  Individuals  can  be  assigned  as  information 
stores  because  of  their  personal  expertise  or  through  circumstantial 


The  Cognitive  Domain  79 


knowledge  responsibility.  Each  individual  participating  in  the 
transactive  memory  has  a  set  of  memory  components.  These 
memory  components  capture  the  key  elements  of  the  collaboration. 
They  represent  information  that  some  individuals  store  externally  in 
other  individuals  and  that  some  individuals  retain  on  behalf  of  other 
individuals  in  the  transactive  memory  system.  The  links  between  an 
individual  and  retrieval  of  a  memory  item  may  be  either  direct  or 
indirect,  taking  “hops”  through  the  transactive  memory  system  until 
the  memory  item  is  accessed. 

As  participants  develop  stronger  relationships  with  other  partici¬ 
pants  through  repeated  or  continued  team  interaction,  the  links 
between  the  participants  become  stronger.  This  suggested  a  second 
common  ground  hypothesis:  The  completeness  of  the  system  for 
recording  and  retrieving  information  depends  on  how  frequently  the 
team  has  recently  collaborated.  This  concept  was  referred  to  as 
“team  hardness.” 

A  time-dependent  functional  model  for  team  hardness  was  defined 
as  0  <  TM(T)  <  1,  and  T  =  x+  1,  where  t  represents  the  time  elapsed 
since  the  start  of  the  operation,  and  r  represents  the  length  of  time 
the  team  has  been  training  or  operating  together.  An  example  is 
included  in  Appendix  B. 

Consensus  plays  a  central  role  in  developing  a  transactive  memory 
system.  It  is  the  majority  opinion  of  a  team  arrived  at  through  active 
collaboration.  Its  definition  implies  the  existence  of  shared  situa¬ 
tional  awareness.  Noting  that  not  all  collaborating  individuals  have 
to  agree  before  a  decision  and  subsequent  action  can  take  place,  we 
are  interested  in  a  measure  of  the  degree  of  consensus.  We  hypothe¬ 
sized  that  the  degree  of  consensus  can  be  estimated  by  the  number 
of  pairwise  combinations  of  collaborating  individuals  who  interpret 
feature  vectors  similarly. 

Models  of  shared  situational  awareness  integrate  our  earlier  model¬ 
ing.  First,  we  placed  the  individual  in  a  team  and  measured  his  situa¬ 
tional  awareness  in  a  team  setting.  Note  that  this  is  not  the  same  as 
team  awareness  but  rather  the  effect  of  team  dynamics  on  an  indi¬ 
vidual  member  of  a  collaborative  decisionmaking  process.  The 
contribution  is  essentially  derivative  of  the  transactional  memory 
function  and,  therefore,  of  team  hardness.  Second,  we  addressed  the 
consensus  that  develops  among  collaborating  individuals  and  its 
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influence  on  the  team’s  shared  situational  awareness.  Finally,  we 
accounted  for  the  diversity  of  decision  agent  capabilities  among  the 
collaborators  that  results  in  our  composite  model  for  the  degree  of 
shared  situational  awareness. 

Implications 

The  foregoing  analysis  indicates  that  for  a  given  team,  the  time  to 
accomplish  a  task  using  collaboration  decreases  as  team  hardness 
increases.  This  squares  with  intuition  and  is  further  supported  by 
Fred  Brooks’  assertion  that  the  time  to  complete  a  software  project 
varies  with  the  degree  of  interaction  necessary  during  the  project 
(Brooks,  1995,  pp.  15-19).  The  more  interaction  required,  the  longer 
the  process,  and  therefore  the  greater  the  need  for  an  experienced 
team,  as  Figure  4.7  illustrates. 

For  a  fixed  team  size  of  n,  the  figure  illustrates  what  happens  when 
teams  that  possess  varying  levels  of  experience  (hardness)  tackle 
tasks  with  complex  interrelationships.  As  the  number  of  people  in 
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Figure  4.7 — Team  Size  and  Hardness  Determine  Task  Duration 
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the  team  increases,  the  time  required  to  complete  the  task  decreases, 
but  at  varying  rates.  At  some  point,  the  time  required  for  the  teams  to 
complete  the  task  increases  when  additional  people  are  added  to  the 
team.  As  with  the  decreasing  rate,  the  increasing  rate  varies  with  the 
team’s  experience. 

Teams  with  assumed  common  ground  or  assumed  common  knowl¬ 
edge  require  additional  interaction  to  transition  to  actual  common 
ground  or  actual  common  knowledge,  as  Figure  4.5  illustrated. 
Teams  that  are  not  hardened,  as  Figure  4.6  illustrated,  have  incom¬ 
plete  transactive  memory  systems  and  simple,  rather  than  complex, 
transactive  memory  systems.  Given  tasks  with  complex  interrelation¬ 
ships,  a  team  with  a  low  degree  of  shared  situational  awareness  will 
require  longer  to  complete  the  tasks  than  a  team  with  a  high  degree 
of  shared  situational  awareness.  Given  that  this  chapter  focused  on 
the  cognitive  process  that  transforms  the  CROP  into  a  decision  and 
subsequent  action,  we  note  that  teams  with  low  degree  of  shared  sit¬ 
uational  awareness  may  be  unable  to  reach  the  decision  point  in  the 
time  available.  Clark  and  Brennan  (1991)  assessed  the  costs  of 
grounding,  noting  1 1  factors  that  contribute  to  achieving  common 
ground.  Further  research  to  relate  these  proposed  costs  of  grounding 
to  the  time  taken  for  a  team  to  accomplish  a  task,  given  its  degree  of 
shared  situational  awareness,  is  warranted.  Also,  the  potential  influ¬ 
ence  of  training  on  how  to  collaborate  and  enabling  information 
technology  needs  to  be  examined. 


Chapter  Five 

FUTURE  WORK 


It  is  error  only,  and  not  truth,  that  shrinks  from  inquiry. 

— Thomas  Paine 


The  preceding  chapters  described  a  mathematical  framework  that 
might  be  used  to  develop  detailed  mathematical  quantities  that  rep¬ 
resent  what  are  generally  considered  to  be  qualitative  concepts.  In 
some  cases,  data  may  exist  in  the  military  C4ISR  community  to  con¬ 
firm  or  disconfirm  both  the  process  and  the  examples  presented  in 
Appendix  B.  In  these  cases,  it  will  be  necessary  to  locate  and  assess 
the  data.  Where  data  do  not  exist,  further  experimentation  or  histori¬ 
cal  analysis  will  be  necessary. 

As  noted  earlier,  the  discussion  of  the  cognitive  domain  is  not  com¬ 
plete.  The  relationship  between  information  quality  and  situational 
awareness  is  the  first  step  in  the  decisionmaking  process.  Further 
work  is  needed  to  codify  the  relationship  between  situational  aware¬ 
ness  and  the  ability  of  the  decisionmaker  to  make  inferences  from 
the  CROP,  that  is,  his  understanding  of  the  situation. 

REFINEMENT  OF  CURRENT  RESEARCH 

The  results  reported  in  this  document  are  not  complete.  The  exam¬ 
ples  used  to  illustrate  the  process  in  Appendix  B  are  notional  and, 
although  intuitively  consistent,  unsupported.  The  mathematical 
framework  also  needs  testing  as  more  evidence-supported  metrics 
are  developed.  Finally,  we  have  focused  almost  exclusively  on  the 
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completeness  measure  of  information  quality.  The  same  emphasis 
needs  to  be  placed  on  correctness,  currency,  and  an  aggregate  mea¬ 
sure.  In  addition,  more  work  is  required  to  reflect  the  influence  of 
collaboration  on  shared  situational  awareness  adequately. 

Data  Fitting 

The  objective  of  this  task  is  to  use  existing  data  to  either  confirm  the 
validity  of  the  relationships  suggested  in  this  work  or  to  suggest  dif¬ 
ferent  relationships.  The  C4ISR  community  could  be  polled  for  data 
that  was  generated  experimentally,  through  exercises,  or  from  perti¬ 
nent  military  operations.  Standard  curve-fitting  techniques  and/or 
statistical  “goodness-of-fit”  analyses  could  be  used,  where  applica¬ 
ble,  to  develop  new  metrics  within  this  report’s  mathematical 
framework.  Of  particular  interest  would  be  collecting  cognitive 
domain  data  that  could  be  used  to  gauge  the  functions  described  in 
the  situational  awareness  portions  of  the  framework. 

Experimentation 

Another  approach  is  to  conduct  experiments  to  gain  additional 
insights  about  information  quality  and  awareness.  Experimentation 
is  attractive  because  several  parameters  can  be  controlled.  For 
example,  decisionmakers  can  be  selected  who  have  various  combi¬ 
nations  of  the  awareness  characteristics  described  in  Table  4.1,  or 
some  set  of  characteristics  that  reflect  recent  research  more  closely. 
Familiarity  with  the  situation  can  be  controlled,  as  can  collaboration. 
Experience  is  a  function  of  the  combat  command  positions  held.  The 
problems  might  be  in  the  form  of  situation  assessments  (CROPs) 
presented  to  the  participants  in  various  ways.  The  degree  to  which 
participants  are  able  to  realize  enemy  intent  from  what  is  presented 
would  then  indicate  their  level  of  awareness. 

Decision,  Understanding,  and  Action 

The  links  between  awareness  and  understanding  and  between 
understanding  and  decision  must  be  established.  The  level  of  aware¬ 
ness  affects  the  ability  to  understand,  i.e.,  to  draw  inferences  about 
the  CROP,  such  as  enemy  intent.  The  inferences  in  turn  affect  the 
decision  to  be  taken  and  therefore  the  subsequent  actions  ordered. 
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The  work  presented  in  this  report  could  be  taken  a  step  further.  As 
stated  at  the  beginning,  the  overall  goal  is  to  establish  a  relationship 
between  information  and  combat  outcomes.  We  postulated  that 
high-quality  information,  relevant  to  the  current  situation,  is  the  key 
to  good  decisions.  The  C4ISR  system,  from  sensors  to  fusion  process¬ 
ing  facilities  and  algorithms,  produces  the  needed  information. 
Making  the  connection  between  information  and  combat  outcomes 
therefore  provides  an  excellent  tool  for  measuring  the  added  value  of 
improved  C4ISR  systems.  Doing  so  will  require  modeling  the  links 
not  just  from  information  to  awareness  (as  done  in  this  report),  but 
from  awareness  to  understanding  and  from  understanding  to  deci¬ 
sion. 

Historical  Analyses 

Past  battles  are  an  important  source  of  information  on  the  value  of 
information.  A  considerable  amount  of  data  is  available  from  various 
sources  that  can  provide  insights  into  the  relationship  between  the 
quality  of  information  and  the  level  of  awareness.  For  several  past 
battles,  information  concerning  “what  each  side  knew’’  is  available  in 
the  form  of  intelligence  reports.  The  quality  of  the  information  repre¬ 
sented  in  the  reports — as  measured  by  timeliness,  completeness,  and 
correctness — could  be  assessed,  given  that  ground  truth  is  known. 
Since  the  outcome  of  the  engagements  is  also  known,  the  relation¬ 
ship  between  the  quality  of  information  and  combat  outcome  can  be 
evaluated. 

Gaming 

It  is  possible  to  use  game  theory  to  illustrate  certain  effects  based  on 
information  imbalances  between  two  opponents.  Recent  RAND 
research  illustrated  the  effects  of  information  superiority  using  a 
simple  two-sided  zero-sum  game  in  which  varying  numbers  of 
strategies  were  available  to  both  sides  (Darilek  et  al.,  2001).  A  series  of 
four  games  were  run,  with  combat  power  the  same  for  both  sides. 
The  relative  information  available  to  each  side  was  varied  in  each 
case.  The  outcomes  illustrated  the  considerable  advantage  gained  by 
the  side  with  information  superiority. 

The  same  type  of  analysis  could  illustrate  the  effects  of  quality  of 
information  on  the  level  of  awareness.  In  a  two-sided  game,  each 
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side  strives  to  obtain  high-quality  information,  in  the  sense  that  it 
should  be  complete,  correct,  and  timely.  At  the  same  time,  each  side 
attempts  to  ensure  that  the  opponent’s  information  is  of  low  quality. 
This  can  be  achieved  through  various  deceptive  practices  and 
through  direct  action  against  information- gathering  and  -processing 
assets.  Several  pairings  of  players  with  varying  awareness  character¬ 
istics  are  then  played  against  the  various  information  quality  levels. 
The  minimax  criterion  could  be  used  to  evaluate  the  outcome  of  the 
game,  thus  establishing  a  link  from  information  quality  to  awareness 
to  decision  and,  finally,  to  outcome. 

APPLICATION  OF  THE  RESEARCH  TO  OTHER  C4ISR 
ARCHITECTURES 

Another  area  for  future  research  would  be  to  apply  the  mathematical 
framework  in  this  report  to  C4ISR  architectures  beyond  the  one  we 
have  discussed.  The  architecture  we  have  described  here  is  a  largely 
automated  system  in  which  sensors  automatically  forward  data  to  a 
centralized  fusion  array,  which  then  pushes  observed  CROPs  to  the 
users.  Analyzing  other  proposed  architectures,  such  as  posting  and 
retrieving  systems  (in  which  sensors  and  users  decide  what  data  to 
post  and  what  to  receive),  and  broadcast  and  subscription  systems 
(in  which  ah  data  are  broadcast  over  the  network,  and  different 
nodes  and  users  decide  what  types  of  data  to  “tune  in’’)  would  be  of 
significant  interest. 


Appendix  A 

SOME  DEFINITIONS 


All  arts  acknowledge  that  then  only  we  know  certainly, 
when  we  can  define;  for  definition  is  that  which  refines  the 
pure  essence  of  things  from  the  circumstance. 

— John  Milton 


This  appendix  records  the  formal  definitions  of  important  terms  and 
concepts  used  in  the  report.  Not  all  the  definitions  listed  are  agreed 
to  by  the  analytic  community  involved  in  the  study  of  C4ISR  pro¬ 
cesses  and  procedures.  In  particular,  such  terms  as  information 
superiority,  data,  and  knowledge  lack  agreed  definitions.  To  the 
extent  possible,  however,  we  have  adopted  standard  definitions. 
However,  the  task  of  the  ISMWG  is  to  develop  accepted  definitions 
for  all  the  terms  used  in  this  report,  and  to  the  extent  that  their  def¬ 
initions  have  been  settled,  we  have  adopted  them  here.1  Other  defi¬ 
nitions  serve  the  purpose  of  ensuring  internal  consistency. 

MEASURES  AND  METRICS 

This  research  focused  on  measures  and  metrics  of  information  qual¬ 
ity  in  the  decisionmaking  process;  it  is  therefore  important  that  we 
include  definitions  of  both.  At  one  level,  measure  theory  offers  strict 
mathematical  definitions  for  measures  and  metrics  that  rely  heavily 


1The  NATO  Panel  on  Best  C4ISR  Practices  has  proposed  definitions  for  some  of  these 
terms.  In  addition,  Joint  publications  and  the  Institute  of  Electrical  and  Electronics 
Engineers  have  offered  definitions  in  various  publications  referenced  in  the  text. 
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on  an  understanding  of  mathematical  analysis  (see,  for  example, 
Halmos,  1950).  Norman  Campbell  suggested  a  less  formal  but  more 
philosophical  approach.  In  a  1921  book  called  What  Is  Science,  he 
argued  that  the  “measurable  properties  of  a  body  are  those  which  are 
changed  by  the  combination  of  similar  bodies.”2  For  example,  the 
weight  and  cost  of  a  bag  of  potatoes  will  change  if  combined  with  a 
second  bag.  However,  the  variety  and  cooking  qualities  of  the  pota¬ 
toes  will  not.  For  Campbell,  then,  “measurement  is  the  representa¬ 
tion  of  properties  by  ‘numbers.’”  In  this  work,  however,  we  offer 
operational  definitions  that  are  more  useful  to  our  purpose.  In  prac¬ 
tice,  measures  and  metrics  are  sometime  used  interchangeably,  and, 
in  many  instances,  this  causes  no  problems.  However,  we  chose  to 
distinguish  between  the  two  and  therefore  illustrate  the  difference 
between  them  and  also  how  they  are  related. 

For  this  work,  we  adopted  a  practical  and  simple  definition  for  mea¬ 
sure-.  a  basis  or  standard  of  comparison.  By  this  definition,  for  exam¬ 
ple,  cost,  distance,  power,  and  brilliance  are  measures  in  that  they 
are  standards  by  which  alternatives  might  be  compared.  Measures 
are  not  always  characterized  by  a  single  standard;  they  may  consist  of 
more-complicated  expressions  for  which  the  associated  yardstick  is 
not  clear,  such  as  “the  degree  to  which  all  unit/target  features  are 
distributed  to  all  users.”  The  yardstick  (metric)  used  to  compare 
alternatives  based  on  this  measure  must  consist  of  some  assessment 
of  “degree”  (percent,  for  example)  and  a  clear  definition  of  what  is 
meant  by  “distributed  to  all  users.” 

A  metric  is  a  scale  used  to  assess  an  alternative’s  position  for  a  given 
measure.  A  metric  is  generally  defined  on  the  real  number  line — 
dollars,  feet,  watts,  candlepower,  etc.  The  mathematical  concept  is 
that  of  a  “distance”  or  the  amount  of  separation  between  two 
“points”  or  alternatives.  For  example,  if  x1  and  x2  are  points  on  the 
real  number  line,  then  the  distance  between  the  two  is  -  x2|.  Ifxt 
and  x2  are  nonnegative  and  if  x1  >  x2,  the  distance  also  measures  how 
much  further  x1  is  than  x2.3  By  convention,  metrics  in  this  report  vary 
from  0  to  1,  with  1  equal  to  the  “best”  value  and  0  to  the  “worst.” 


2The  selection  quoted  above  is  taken  from  excerpts  reprinted  in  Neuman  (1988), 
p.  1,772. 

3A  formal  axiomatic  definition  of  metrics  and  metric  spaces  can  be  found  in  Kreyzig 
(1978). 
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INFORMATION 

The  following  terms  are  fundamental  to  a  theory  about  how  infor¬ 
mation  affects  human  and  organizational  behavior.  Although  the 
following  are  attempts  to  record  precise  definitions,  we  recognize 
that  many  of  these  terms  have  no  agreed- on  operational  definition  as 
yet.  However,  to  the  extent  possible,  we  selected  definitions  that  are 
consistent  with  the  information  “primitives”  discussed  in  Alberts  et 
al.  (2001). 4  We  defined  these  terms  as  follows: 

•  Data:  any  representation,  such  as  characters  or  analog  quanti¬ 
ties,  to  which  meaning  might  be  assigned.  Examples  include 
radar  returns,  infrared  detections,  and  sensor  reports. 

•  Information:  data  that  have  been  processed  in  some  way  to  put 
the  individual  data  elements  into  some  meaningful  context.  One 
example  would  be  the  transformation  of  a  radar  return  from  a 
GMTI  sensor  into  the  location  and  velocity  of  a  moving  vehicle. 

•  Knowledge :  conclusions  that  are  drawn  from  the  accumulated 
information  based  on  patterns  or  some  other  basis  for  gaining 
insights  constitute  knowledge.  For  example,  we  “know”  a  large 
armored  unit  is  traveling  toward  our  position  in  a  westerly  direc¬ 
tion  because  of  GMTI  returns  and  intelligence  reports. 

•  Situational  awareness:  a  realization  of  the  current  situation 
constitutes  situational  awareness  and  results  from  the  interac¬ 
tions  between  prior  knowledge,  beliefs,  and  the  current  percep¬ 
tion  of  reality.  For  example,  it  is  possible  to  infer  that  the  enemy 
will  attack  from  the  west  based  on  (1)  current  knowledge  that  a 
large  armored  unit  is  traveling  toward  us  in  a  westerly  direction; 
(2)  prior  knowledge  of  the  enemy  commander’s  behavior  under 
similar  circumstances;  and  (3)  belief,  arising  from  years  of  train¬ 
ing,  that  similarly  configured,  rapidly  moving  armored  columns 
lead  to  an  enemy  attack.5  In  the  text,  awareness  and  situational 
awareness  are  sometimes  used  interchangeably. 


4The  complete  set  of  primitives  consists  of  sensing,  information,  knowledge,  aware¬ 
ness,  understanding,  decisions,  actions,  information  sharing,  shared  knowledge, 
shared  awareness,  collaboration,  and  synchronization.  See  Alberts  et  al.  (2001). 

5Other  definitions  of  situational  awareness  also  focus  on  a  state-of-mind  definition. 
Carl  Builder  referred  to  situational  awareness  as  a  state  attained  by  a  decisionmaker  in 
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•  Understanding:  the  ability  of  humans  to  draw  inferences  about 
the  possible  current  and  future  consequences  of  a  situation — the 
process  of  comprehending,  of  appreciating  the  meaning  of  a 
word,  sentence,  event,  proposition,  map  display,  etc.  Current 
and  past  knowledge  lead  to  inferences  about  the  current  situa¬ 
tion,  while  understanding  how  events  might  unfold  is  a  function 
of  an  awareness  of  the  situation.  When  attempting  to  assess  the 
situation,  understanding  produces  a  higher  level  of  inferential 
capability  and  is  sometimes  referred  to  as  sense  making. 

Clearly,  the  boundaries  between  each  of  the  terms  defined  above  are 
situation  dependent.  For  example,  according  to  the  definition  above, 
a  track  established  through  several  observations  of  a  moving  column 
is  information.  However,  if  it  is  one  of  several  tracks  constituting  a 
total  enemy  force,  it  may  be  a  single  data  point  in  the  friendly  com¬ 
mander’s  attempt  to  estimate  when  and  where  the  enemy  force  will 
attack. 

INFORMATION  SUPERIORITY 

Joint  Publication  1-02  formally  defines  information  superiority  to  be: 
“  [t]  hat  degree  of  dominance  in  the  information  domain  which 
permits  the  conduct  of  operations  without  effective  opposition” 
(DoD,  2003,  p.  255).  However,  information  superiority  is  really  more 
than  that  because  it  closely  aligns  with  information  requirements. 
Asymmetric  information  needs  are  as  much  a  part  of  determining 
information  superiority  as  the  ability  to  collect,  process,  and  dissemi¬ 
nate  an  uninterrupted  flow  of  information  is.  Moreover,  information 
is  never  constant  on  a  high-tempo  battlefield,  and  needs  will  there¬ 
fore  vary. 

The  official  definition  therefore  requires  answers  for  two  important 
questions: 


which  he  is  cognizant  of  the  key  physical,  geographical,  and  meteorological  features  of 
the  battlespace  that  will  enable  his  command  concept  to  be  realized  (Builder,  Bankes, 
and  Nordin,  1999,  p.  xv).  Some  researchers  have  referred  to  this  broader  definition  as 
sense  making,  thus  emphasizing  the  close  link  between  situational  awareness  and 
“understanding.”  See  the  glossary  for  a  definition  of  the  latter,  as  set  forth  by  Smith 
(1999). 
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•  What  are  the  relative  information  needs  of  both  sides? 

•  Under  what  conditions  do  those  needs  change? 

From  a  practical  standpoint,  what  is  needed  is  knowledge  of  what 
each  side  tries  to  collect,  process,  and  disseminate  and  how  this  will 
change  as  the  battle  progresses.  The  definition  therefore  requires 
knowledge  of  the  collection,  processing,  and  dissemination  regimes 
for  both  sides.  If  the  friendly  commander  is  able  to  (1)  collect,  pro¬ 
cess,  and  disseminate  information  when  needed;  (2)  understand  how 
his  adversary’s  information  needs  will  change  over  time;  and  (3)  stop 
the  enemy  from  collecting,  processing,  and  disseminating  informa¬ 
tion,  he  can  claim  information  superiority.  This  means  that  both 
sides  can  be  expected  to  engage  in  information  operations.  Both 
sides  must  be  capable  of  physically  and  electronically  protecting 
their  C4ISR  assets  and  can  be  expected  to  concentrate  some  of  their 
resources  on  attacking  the  other’s  C4ISR  assets.  This  all  suggests  an 
expanded  definition  of  information  superiority: 

Information  superiority :  A  side  has  information  superiority  when  it 
is  able  to  (1)  collect,  process,  and  disseminate  information  as 
needed;  (2)  anticipate  the  changes  in  its  own  and  its  adversary’s 
information  needs;  and  (3)  deny  its  adversary  the  ability  to  do  the 
same. 

NETWORK-CENTRIC  WARFARE 

It  is  impossible  to  ignore  the  compelling  arguments  made  for  the 
contribution  of  NCW  to  combat  outcomes.  The  dual  nature  of  the 
concept,  networks  and  improved  command  and  control  procedures, 
makes  it  central  to  any  discussion  about  the  quality  of  information 
and  decisionmaking — particularly  when  assessing  their  influence  on 
combat  operations. 

NCW  can  be  defined,  according  to  Alberts,  Garstka,  and  Stein  (2000, 
p.  2),  as 

an  information  superiority-enabled  concept  of  operations  that  gen¬ 
erates  increased  combat  power  by  networking  sensors,  decision 
makers,  and  shooters  to  achieve  shared  awareness,  increased  speed 
of  command,  higher  tempo  of  operations,  greater  lethality, 
increased  survivability,  and  a  degree  of  self- synchronization.  In 
essence,  NCW  translates  superiority  information  superiority  into 
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combat  power  by  effectively  linking  knowledgeable  entities  in  the 
battlespace. 

In  contrast  to  network-centric  operations  or  warfare,  traditional  war¬ 
fare  is  considered  to  be  platform  centric.  The  difference  between  the 
two  is  that,  in  platform- centric  warfare,  one  must  mass  force  to  mass 
combat  effectiveness  by  physically  massing  platforms  (weapon  sys¬ 
tems)  to  achieve  combat  effects;  in  NCW,  combat  effects  are 
achieved  by  virtually  massing  forces  through  a  robust  network.  That 
is,  the  employment  of  weapon  systems  on  the  battlefield  is  optimized 
so  that  a  target  is  serviced  by  the  most  effective  system  in  the  net¬ 
work.  Thus,  the  hypothesis  is  that  a  much  smaller  force  can  provide 
the  same  effects  as  massing  force.  This  is  particularly  advantageous 
in  the  face  of  declining  defense  budgets,  allowing  commanders  to  get 
more  from  less. 

NCW  is  derived  from  a  concept  from  the  business  community,  the 
most  notable  example  of  which  is  Wal-Mart’s  network- centric 
retailing.  In  Wal-Mart’s  platform-centric  days,  it  had  a  central 
purchasing  department  that  relied  on  inventory  status  reports  from 
its  various  stores.  This  was  replaced  with  point-of-sale  scanners  and 
inventory  sensors  that  order  replacements  directly  from  the  vendor 
as  needed.  The  automated  system  also  produces  sales  statistics  on 
thousands  of  items,  in  real  time,  and  marketing  decisions  are  made 
accordingly,  in  near  real  time.  Information  on  inventory,  deliveries, 
and  sales  for  any  period  of  time  is  available  to  each  store  manager. 
The  Navy  and  the  C4ISR  community  are  pursuing  the  application  of 
this  concept  to  warfare  (Cebrowski,  1998). 

NCW  employs  the  concept  of  three  network  grids: 

1.  The  information  network  grid:  The  information  grid  provides  the 
infrastructure  to  “receive,  process,  transport,  store,  and  protect 
information  for  the  foint  and  combined  services”  (Stein,  1998). 
This  is  therefore  the  network- centric  parallel  to  the  information 
domain. 

2.  The  sensor  network  grid:  The  sensor  grid  is  a  need-based  network 
that  makes  use  of  the  sensors  in  the  information  grid  that  are  per¬ 
tinent  to  a  given  task.  It  consists  of  such  typical  warfare  sensors  as 
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radar  and  imbedded  logistics  sensors  to  track  supply.  The  sensor 
grid  is  unique  to  each  task. 

3.  The  engagement-decision-shooter:  Sensor  and  warfighter  ele¬ 
ments  of  the  network  are  tasked  to  attack  in  the  engagement  grid. 
This  grid,  like  the  sensor  grid,  is  dynamic,  using  a  unique  blend  of 
warfighters  and  sensors  for  each  new  task  (Stein,  1998). 

The  second  and  third  grids  are  completely  contained  in  the  informa¬ 
tion  network  grid.  Each  grid  is  composed  of  nodes  represented  by 
individual  sensors,  weapons,  or  command  platforms  and  is  con¬ 
nected  via  networked  data  and  communications.  The  sensor  and 
engagement  grids  are  not  necessarily  separate,  often  having  overlap¬ 
ping  components.  For  example,  the  sensor  grid  begins  a  track  on  a 
cruise  missile  and  continues  to  track  as  the  pertinent  unit  engages 
and  a  kill  is  made. 

NCW  flattens  the  command  and  control  pyramid.  Commanders 
communicate  intent  by  introducing  doctrine  in  the  form  of  com¬ 
puter  algorithms  and  by  communicating  directly  with  individual 
units.  NCW  moves  toward  automated  optimization  of  the  positions 
of  units  in  a  group  and  engagement  of  enemy  forces  using  new  ini¬ 
tiatives,  such  as  the  Navy’s  Cooperative  Engagement  Capability 
(CEC)6  and  Ring  of  Fire.7 

Although  NCW  is  not  about  networks  exclusively,  communications 
networks  play  a  major  role  in  achieving  its  goals.  In  a  recently  pub¬ 
lished  book  on  the  subject,  Alberts,  Garstka,  and  Stein  (1999,  p.  12) 
state  that  Network  Centric  Warfare  is  about 

exploiting  information  to  maximize  combat  power  by  bringing 
more  of  our  available  information  and  warfighting  assets  to  bear 
more  effectively  and  efficiently.  [It  is]  about  developing  collabora- 


6CEC  is  designed  to  combine  the  raw  sensor  data  from  all  platforms  involved  in  an 
operation,  regardless  of  age  or  type  of  sensors  on  individual  platforms.  It  allows  the 
combined  data  from  these  sources  to  produce  a  more  complete,  shared  COP  for 
tracking  purposes.  For  additional  information,  see  “The  Cooperative  Engagement 
Capability”  (1995). 

7The  Ring  of  Fire  (ROF)  concept  is  a  network-centric  approach  to  littoral  warfare.  It 
links  land,  sea,  and  air  forces  to  produce  calls  for  fire.  Like  the  CEC,  ROF  networks  sen¬ 
sor  and  weapon  information  for  sea,  shore,  and  command  forces  in  the  littoral  to  pro¬ 
duce  an  extended  and  more  accurate  COP  (see  Mitchell,  1998). 
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tive  working  environments  for  commanders,  and  indeed  for  all  our 
soldiers,  sailors,  marines  and  airmen  to  make  it  easier  to  develop 
common  perceptions  of  the  situation  and  achieve  (self-)  coordi¬ 
nated  responses  to  situations. 

Exploiting  information  requires  the  ability  to  distribute  fused  infor¬ 
mation  rapidly  to  all  participating  command  nodes.  This  implies  a 
robust  communications  network,  as  does  the  requirement  for  a  col¬ 
laborative  working  environment.  One  of  the  benefits  of  NCW  is 
increased  shared  situational  awareness  of  the  battlespace. 

COGNITIVE  DOMAIN  TERMS 

The  terms  situational  awareness,  shared  situational  awareness, 
cognition,  understanding,  and  collaboration  appear  frequently  in 
discussions  of  combat  decisionmaking.  We  defined  understanding 
and  situational  awareness  above,  under  the  information  domain, 
where  they  are  also  important  concepts.  Formal  definitions  of  the 
remaining  terms  exist  but  are  not  always  consistent  and  are 
sometimes  not  precise  enough  to  satisfy  the  requirements  of 
rigorous  mathematical  analysis.  We  tailored  the  following  specific 
definitions  to  meet  the  requirements  of  this  research: 

•  Shared  situational  awareness :  the  ability  of  a  decisionmaking 
team  to  share  their  realization  of  the  current  situation.  Note  that 
based  on  this  definition,  consensus  among  the  team  members 
implies  shared  situational  awareness.8 

•  Collaboration-,  a  process  in  which  two  or  more  people  work 
together  to  achieve  a  common  objective.  Collaboration  requires 
the  ability  to  share  information. 

•  Cognition-,  the  ability  of  humans  to  derive  certain  knowledge 
from  an  information  source,  such  as  a  CROP.  Note  that  cog¬ 
nizance  does  not  imply  either  understanding  or  awareness. 


8The  Army’s  Digitization  Office  defines  shared  situational  awareness  as  "the  ability  of 
a  unit  to  know  where  its  friends  are  located,  where  the  enemy  is,  and  to  share  that 
information  with  other  friends,  both  horizontally  and  vertically,  in  near  real-time.” 
Again,  this  is  a  state-focused  definition.  It  describes  the  state  required  to  achieve 
shared  situational  awareness  (U.S.  Army  Digitization  Office,  1994). 
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However,  the  reverse  is  true,  i.e.,  both  awareness  and  under¬ 
standing  imply  cognizance. 

Common  ground:  the  knowledge,  beliefs,  and  suppositions  that 
individual  collaborating  team  members  believe  they  share. 
Suppositions  are  the  propositions  whose  truths  a  team  member 
takes  for  granted. 

Belief:  a  proposition  that  a  participant  would  assent  to  if  given 
ample  opportunity  to  reflect.  Along  with  facts  and  assumptions, 
beliefs  constitute  the  foundation  of  common  ground. 
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Few  things  are  harder  to  put  up  with  than  the  annoyance 
of  a  good  example. 

— Mark  Twain 


In  the  main  text,  we  mentioned  specific  mathematical  constructs 
that  might  be  used  to  instantiate  the  general  framework.  This 
appendix  offers  a  few  of  these,  in  addition  to  some  more  general 
methods  mentioned  in  the  text.  It  is  important  to  understand  that 
these  expressions  are  purely  speculative  and  are  not  the  results  of 
any  in-depth  research.  However,  they  do  illustrate  the  fact  that  rea¬ 
sonable  mathematical  expressions  can  be  developed  to  measure  the 
quality  of  information  throughout  the  information  superiority  value 
chain. 

SIMILARITY  MATRICES 

Similarity  matrices  serve  to  quantify  generally  nonquantifiable  fea¬ 
tures  by  examining  the  “closeness”  of  the  attributes.  For  example, 
suppose  unit  type  is  the  feature  to  be  assessed  for  a  land  force  and 
that  the  complete  set  of  possible  attributes  of  unit  type  is  U  = 
[Armored,  Artillery,  Mechanized,  Reconnaissance],  Further  assume 
that  one  way  to  characterize  each  type  is  by  the  equipment  it  usually 
possesses.  For  example,  suppose  that  the  complete  set  of  major 
equipment  types  that  characterizes  each  of  these  units  is  tanks, 
armored  fighting  vehicles,  armored  personnel  carriers,  and  wheeled 
vehicles.  Table  B.l  offers  a  notional  distribution  of  equipment  for 
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each  of  these  unit  types  in  terms  of  the  fraction  of  the  total 
equipment  in  the  unit. 

A  table,  such  as  Table  B.l,  can  be  transformed  into  a  similarity 
matrix.  Similarity  or  dissimilarity  may  be  defined  in  terms  of  a  dis¬ 
tance  metric,  such  as  the  Euclidean  distance.  The  following  example 
demonstrates  how  Table  B.l  might  be  transformed  into  a  similarity 
matrix. 

Suppose  we  wish  to  calculate  the  “distance”  between  an  armored 
unit  and  a  mechanized  unit.  Using  the  Euclidean  distance,  we  first 
calculate  the  “dissimilarity”  as1: 

D armor, mech  =  ^0.23  -  O.to)2  +  (o,19  -  0.3o)2  +  (o.38  -  0.3o)2  +  (o,19  -0.3())2 
=  0.218  . 


Table  B.l 

Distribution  of  Equipment  by  Unit  Type 


Tanks 

(%) 

Armored 

Fighting 

Vehicles 

(%) 

Armored 

Personnel 

Carriers 

(%) 

Wheeled 

Vehicles 

(%) 

Armor 

0.231 

0.192 

0.385 

0.192 

Artillery 

0.000 

0.000 

0.000 

1.000 

Mechanized 

0.100 

0.300 

0.300 

0.300 

Reconnaissance 

0.000 

0.000 

0.250 

0.750 

1Note  that  the  maximum  distance  between  any  two  units  is  the  square  root  of  two. 
This  derives  from  the  fact  that  the  fractional  equipment  counts  must  sum  to  1.0.  The 
extreme  case  is  two  units,  each  consisting  of  only  one  type  of  equipment,  which  is  dif¬ 
ferent  for  each  unit.  For  example,  the  distance  between  two  units,  one  consisting  only 
of  tanks  and  the  other  only  of  armored  personnel  carriers,  would  be 

Dab  =  J(  l.o  -  o.o)2  +  (o.o  -  i.o)2  +  (o.o  -  o.o)2  +  (o.o  -  o.o)2  =  V2 . 
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The  degree  to  which  the  two  unit  types  are  similar  is  then  the  com¬ 
plement  of  this  figure:  Sarmor>mech  =  1  -  Darmormech  =  0.782.  This  value 
is  relatively  high  because  armor  and  mechanized  units  are  rather 
similar  in  composition  of  tracked  vehicles.  If  we  do  this  for  each  pair 
of  unit  types,  the  resulting  nonnormalized  similarity  matrix  will  be  as 
shown  in  Table  B.2. 

We  next  must  assess  the  probability  that  a  unit  is  correctly  classified. 
That  is,  what  is  the  probability  that  a  unit  of  type  i  is  not  mistakenly 
classified  as  a  unit  of  type  j?  To  assess  this  likelihood,  we  first  normal¬ 
ize  the  rows  of  the  matrix  in  Table  B.2  to  get  the  matrix  shown  in 
Table  B.3.  The  entries  in  Table  B.3  can  now  be  thought  of  as  condi¬ 
tional  probabilities.  For  example,  the  probability  that  an  armored 
unit  is  correctly  classified,  given  a  report  that  the  unit  is  mechanized, 
is  P(armor|mechanized)  =  0.357. 

Suppose  now  that  we  interpret  the  entries  as  correctness  values  for 
estimates.  That  is,  the  correctness  of  an  estimate  that  the  unit  is 

Table  B.2 

Similarity  Matrix  for  Unit  Type 


Armor 

Artillery 

Mechan¬ 

ized 

Reconnais¬ 

sance 

Armor 

1.000 

0.056 

0.782 

0.352 

Artillery 

0.056 

1.000 

0.175 

1.646 

Mechanized 

0.782 

0.175 

1.000 

0.448 

Reconnaissance 

0.352 

0.646 

0.448 

1.000 

NOTE:  The  diagonal  elements  are  all  1.000  because  all  units  are 
completely  similar  to  themselves. 


Table  B.3 

Normalized  Similarity  Matrix  for  Unit  Type 


Armor 

Artillery 

Mechan¬ 

ized 

Reconnais¬ 

sance 

Armor 

0.4564 

0.0257 

0.3570 

0.1608 

Artillery 

0.0300 

0.5324 

0.0934 

0.3442 

Mechanized 

0.3252 

0.0729 

0.4158 

0.1861 

Reconnaissance 

0.1440 

0.2642 

0.1830 

0.4087 
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mechanized,  given  that  it  is  really  armored,  is  C(mechanized)  =  0.357. 
In  practice,  we  arrive  at  this  estimate  by  observing  n  unit  type  reports 
for  a  single  unit.  If  the  most  frequently  reported  unit  type  is  mecha¬ 
nized,  this  is  the  estimate  we  use.  The  correctness  of  the  estimate, 
however,  is  0.357.  Note  that  this  correctness  estimate  is  based  on  the 
similarity  of  a  mechanized  unit  to  an  armored  unit  and  that  it 
presupposes  that  actual  unit  type  is  known.  If  the  unit  type  is  not 
known,  we  can  assess  the  correctness  of  the  report  by  assuming  that 
the  unit  is  indeed  mechanized.  In  this  case,  we  get  C(mechanized)  = 
0.416. 

The  same  matrix  can  be  used  to  calculate  the  variance  of  the  correct¬ 
ness  of  a  sample  of  reported  detections.  Suppose  we  receive  a  sample 
of  four  reports  of  unit  type  U  =  {armor,  mechanized,  mechanized, 
artillery} . 

The  unit  type  estimate  from  this  set  of  reports  is  “mechanized.”  If  the 
unit  is  in  fact  an  armored  unit,  as  assumed  above,  the  correctness  of 
the  estimate  is  U  =  C(mechanized)  =  0.357,  and  the  variance  in  the 
correctness  is 


lljuj-u)2 

3 

(o.456  -  0.357)2  +  (0.357  -  0.357)2  +  (0.357  -  0.357)2  +  (o.026  -  0.357)2 

3 

0.0398  . 


In  general,  for  n  reports,  the  variance  is 


SET-THEORETIC  TRACKING  METRIC 

If  we  assume  that  the  number  of  active  tracks  at  time  t-  1  is  Xt_  ,  and 
that  at  scan  t  it  is  Xt,  the  question  about  how  well  is  the  sensor  system 
tracking  targets  depends  on  the  relative  size  of  Xt_  xand  Xt.  In  the 
main  report,  we  discussed  the  possibilities  using  some  assumptions. 
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Here,  we  wish  to  expand  a  bit  more  on  the  development  of  the  sug¬ 
gested  tracking  metric. 

Figure  B.l  depicts  a  generalized  representation  of  the  two  sets  of 
tracks.  To  develop  a  metric,  0  <  T  <  1,  we  examined  the  size  of  the 
intersection,  AH  B.  Tracks  from  scan  i  -  1  that  are  still  active  in  scan  t 
reside  in  the  intersection.  The  number  of  tracks  in  A  n  B  is  |  Xf  _  i  -  Xt|. 
An  appropriate  metric  therefore  might  be 


This  metric  satisfies  the  requirements  established  in  the  main  report 
that,  when  A  Pi  B  =  0  {no  previous  tracks  are  active),  T=  0;  when 
AHB-  A  {all  previous  tracks  are  active),  T  =  1;  for  all  other 
possibilities,  0  <  T<  1. 

CRITICAL  PATH  METHOD 

The  CPM  is  used  to  assess  the  time  required  to  complete  a  process 
(see,  for  example,  Taha,  1976).  This  section  illustrates  this  methodol¬ 
ogy.  This  project  management  tool  was  designed  to  identify  tasks  in  a 
project  that  require  special  attention  because  slippage  in  these  will 
cause  slippage  in  the  completion  of  the  project.  The  critical  path 
defines  a  chain  of  critical  activities  that  connect  the  start  and  end  of 
events  in  a  directed  network,  such  as  that  depicted  in  Figure  B.2.  It 
reflects  the  time  required  to  traverse  the  longest  (most  time  consum¬ 
ing)  path  in  the  network  of  tasks. 
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Figure  B.l — Track  Sets 
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Figure  B.2 — Example  Currency  Calculation 


In  assessing  the  currency  of  information,  the  central  issue  is  which 
process  times  recorded  are  sequential  and  which  are  parallel.  For 
example,  sensor  reports  cannot  be  combined  to  produce  the  CROP 
until  the  they  have  been  collected.  The  total  time  elapsed  in  the 
information  domain,  then,  is  either  the  sum  of  the  times  required  in 
each  subdomain,  the  maximum  time  across  subdomains,  or  some 
combination  of  both.  Although  the  actual  calculation  will  depend  on 
the  architecture  designed  for  each  given  alternative,  we  can  illustrate 
the  process  with  an  example. 

Suppose  we  assign  times  to  the  various  tasks  defined  for  each  sub- 
domain.  We  have  not  attempted  to  decompose  these  tasks  into  sub¬ 
tasks,  although  in  practice,  this  is  highly  recommended.  Figure  B.2 
graphically  represents  the  sequencing  for  four  tasks.  The  circles  rep¬ 
resent  events,  and  the  arcs  represent  processes  to  complete  in  the 
time  specified.  The  structure  of  the  graph  is  situation  dependent,  as 
the  discussion  of  the  events  and  the  tasks  will  make  clear. 

Event  1.  This  starts  the  sensing  process.  It  can  be  considered  the  ini¬ 
tial  tasking  to  the  sensor  suite.  Two  tasks  begin  at  once:  Initial  detec¬ 
tion  and  target  tracking  and  retasking  of  sensors  to  capture  high- 
value  targets — if  that  capability  exists. 

Event  2:  Detection  and  tracking  are  complete.  A  better  interpretation 
might  be  that  sufficient  sensor  data  are  available  to  allow  the  fusion 
process  to  begin.  This  is  represented  by  task  3. 
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Event  3:  A  CROP  is  produced.  This  cannot  occur  until  the  develop¬ 
ment  of  the  CROP  is  complete  (task  3).  Note  that  it  does  not  depend 
on  the  retasking.  The  assumption  for  this  example  is  that  retasking 
continues  and  that  updates  are  processed  as  they  arrive  or  at  speci¬ 
fied  intervals. 

Event  4:  This  is  actually  two  events.  The  first  is  the  dissemination  of 
the  CROP  to  the  users,  and  the  second  is  the  completion  of  retasking. 
At  this  point,  the  CROP  is  complete  and  has  been  disseminated  to  the 
users. 

The  composite  system  currency,  given  these  assumptions  concern¬ 
ing  the  processes,  obtained  by  inspection,  is  then 


t=  maxKh  +  t3  +  t4),  t2}. 


A  more  systematic  approach  for  more  complex  task  and  event 
groupings  can  be  found  in  any  standard  operation  research  text. 

SENSORS 

Figure  B.3  illustrates  a  typical  model  of  sensor  detection  performance 
as  a  function  of  range.  A  sensor  with  an  unobstructed  view  of  the 
battlespace  will  generally  have  a  minimum  and  a  maximum  range. 
Within  this  range  band,  the  sensor  is  capable  of  detecting  targets  of  a 
type  specific  to  its  technical  capabilities.  The  maximum  range  will 
vary  according  to  several  factors.  For  example,  the  available  power 
and,  if  the  sensor  is  used  to  classify  targets,  its  resolution  will  affect 
an  active  sensor’s  maximum  range.  The  minimum  range  may  be  zero 
or  may  be  a  significant  distance,  as  for  radar  and  imaging  systems. 

Within  its  operational  range,  a  sensor  will  have  a  nonzero  probability 
of  detecting  its  designated  targets.  In  Figure  B.3,  we  have  the 
following  functional  relationship: 

k  if  dmin  <d<  dh 

f(d)  if  dh  <  d  <  dmax 

0  otherwise . 

The  sensor  achieves  its  maximum  capability,  P[d)  =  k  <  1,  at  the 
minimum  range  and  maintains  that  capability  through  range  db. 
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Figure  B.3 — Generic  Sensor  Performance  Model 

Performance  begins  to  fall  off  beyond  db  according  to  some  relation¬ 
ship  (depicted  as  a  straight  line  in  the  diagram) .  The  functional  rela¬ 
tionship  may  vary  as  a  function  of  a  number  of  factors,  including  tar¬ 
get  aspect  angle  relative  to  the  sensor,  target  state  (temperature, 
velocity,  configuration,  etc.),  and  the  state  of  the  immediate  and 
intervening  environments.  The  generic  profile  in  Figure  B.3  can  be 
used  to  model  the  performance  of  many  sensor  types. 

A  Reliability  Model  of  Completeness 

It  seems  natural  to  equate  the  probability  that  a  sensor  will  detect  a 
target  with  its  reliability.  Because  the  completeness  of  the  reports 
from  a  single  sensor  is  a  function  related  to  its  reliability,  we  might 
choose  sensor  reliability  as  a  metric  for  the  completeness  of  the  sen¬ 
sor  reports.  The  problem  is  then  to  examine  the  nature  of  P{d)  for 
each  sensor  and  for  the  entire  suite.  One  way  to  do  this  is  to  assess 
the  reliability  of  the  sensor  and  the  sensor  suite.  We  let  Pj{d)  =  R^d) 
be  the  reliability  of  sensor  Sit  where  d  is  the  distance  from  the  sensor 
to  the  target.  Generally,  the  independent  variable  is  time,  but  it  need 
not  be.  In  the  case  of  sensors,  as  discussed  above,  distance  is  much 
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more  useful  in  that  it  can  account  for  malpositioned  sensors  and 
inefficient  multisensor  configurations  and  because  it  generally  char¬ 
acterizes  sensor  detection  performance.  The  general  form  of  R{d)  is 


where  r(d)  is  the  failure  rate  function  and  is  dependent  on  the  char¬ 
acteristics  of  the  sensor  and  the  current  operating  situation.  Sensor 
characteristics  include  the  ability  to  detect,  estimate,  and  classify  tar¬ 
gets.  Note  that,  for  d  =  0,  i?(d)  =  1.  That  is,  when  the  sensor  is  coter¬ 
minal  with  the  target,  it  is  infallible.  This  is,  of  course,  an  idealization; 
therefore,  to  conform  more  closely  to  the  generic  model,  we  recast 
the  reliability  function  as  the  decreasing  segment  in  a  piecewise 
relationship  so  that  we  now  have 


I  \  fO  if  d  <  dmin 

=  \  -fd  .  r(s)ds 

[e  mm  otherwise. 

Note  also  that  R(d)  is  a  probability  that  can  be  interpreted  as  the 
probability  of  detection  at  range  d.2  In  this  formulation,  dmax  is 
reached  only  in  the  limit.  This  is  not  much  of  a  problem,  however, 
since  the  rate  of  decline  of  R{d)  for  d  >  dmin  is  controlled  by  r(d). 
Depiction  of  the  linear  decline  is  more  problematic.  In  this  case,  the 
reliability  formulation  should  be  replaced  with 


Rid)  = 


0 


d 


ma x-d 


^max  ^min 


if  d  <  dmin 
otherwise. 


Expressing  sensor  detection  probability  in  this  way  can  be  useful.  For 
example,  the  effects  of  occlusions  can  be  modeled  through  the 
appropriate  selection  of  r(d)  so  that  for  a  totally  occluded  view, 
d  o for  impaired  views,  such  as  foliage,  atmospheric  disturbances, 
etc.,  d  is  set  to  be  larger  than  the  physical  distance  between  the  sen- 


2There  are  several  good  texts  on  reliability  engineering.  See,  for  example,  Ayyub  and 
McCuen  (1997)  and  Pecht  (1995). 
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sor  and  the  target.  This  suggests  that  /'(h) might  be  defined  piecewise 
to  reflect  the  effects  of  occluded  views.  We  pursue  this  more  fully 
below. 

As  an  example,  suppose  we  have  three  sensors  whose  detection 
probabilities  beyond  the  minimum  range,  dmin,  are  determined  by 
the  following  failure  rates:  (s)  =  1,  r2{s)  =  s,  and  r3(s)  =  s2.  This  leads 
to  the  following  detection  probability  functions: 

Rl[d)  =  e^d~dmia\ 

R2(d)  =  e~^2~<in\  and 

Figure  B.4  depicts  varying  detection  probabilities  for  these  three  sen¬ 
sors,  with  hmin  =  0  for  simplicity.  Note  that  the  relative  behavior  of  the 
sensors,  as  well  as  their  absolute  behavior,  is  dependent  on  their 
distance  to  the  target.  Sensor  3,  for  example,  is  preferable  when  the 
target  is  close  in,  but  sensor  1  outperforms  the  other  two  when  the 
target  is  at  a  greater  distance.  From  an  operational  point  of  view, 
therefore,  it  is  important  to  know  where  the  intersection  of  the  per¬ 
formance  lines  is,  as  depicted  in  the  diagram.  This  is  equivalent  to 
knowing  the  relative  performance  of  the  sensors,  within  their  speci¬ 
fied  performance  bands. 

Fiowever  compelling  this  construct  might  be,  it  is  important  to  note 
that  it  ignores  the  importance  of  false  targets.  For  example,  Sensor  3 
may  be  preferable  up  to  1.5  km  in  Figure  B.4,  but  that  may  also  mean 
that  is  more  likely  to  detect  false  targets  at  that  range  if  we  assume 
that  a  sensor  is  as  likely  to  detect  false  as  it  is  to  detect  real  targets. 
The  problem  is  one  of  discrimination.  How  well  can  the  sensor  dif¬ 
ferentiate  real  from  false  targets  and  at  what  ranges? 

Occlusions 

A  sensor  is  occluded  when  terrain  and/or  foliage  intervene  between 
the  sensor  and  the  target.  Most  sensors  require  a  clear,  unobstructed 
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Sensor  3  is  Sensor  2  is  Sensor  1  is 

preferable  preferable  preferable 


Figure  B.4 — Sensor  Performance 


view  of  the  target.  Targets  can  be  hidden  from  view  by  the  terrain 
features,  for  example  mountain  ranges.  The  elevation  profile  of  the 
terrain  in  the  AO  can  be  considered  to  be  a  geometric  surface, 
z  =  g{x,  y),  where  x  and  y  are  planer  coordinates,  such  as  latitude  and 
longitude  or  Universal  Transverse  Mercator  (UTM)  coordinates. 
Figure  B.5  illustrates  a  typical  AO,  in  which  a  sensor  is  located  at 
some  elevation  at  (x0,y0),  and  a  target  is  located  on  the  surface  at 
(Xi,yi).  The  area  within  which  ground  targets  can  be  detected  is 
indicated  in  medium  gray.  The  area  within  which  ground  targets 
cannot  be  detected  is  indicated  in  light  gray.  The  sensor  can  detect 
all  the  targets  within  the  valley  in  the  foreground  but  none  of  those 
beyond  the  mountain  range  (for  example,  those  in  the  next  valley). 
Complex  geometric  relationships  can  exist  between  targets,  sensors, 
and  the  environment. 
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Target 


Sensor 


(*o  >y0) 


x-Longitude 


Figure  B.5 — Terrain  Occluded  Targets 


We  can  also  use  the  reliability  function  to  model  the  probability  of 
detection  for  different  levels  of  occlusion  without  resorting  to  com¬ 
plex  surface  maps.  For  example,  suppose  the  failure  rate  for  a  given 
sensor  without  occlusions  is  r(s)  =  1.  This  produces  a  detection  prob¬ 
ability  function  like  that  for  Sensor  1  above: 


R(d)  =  e^d~dmin^  . 


We  can  now  express  the  effects  of  occlusions  by  damping  R[d),  so 
that 


R(d)  =  ke-d-d' 


min 


where  /cE  [0,1].  For  k  =  0,  we  have  a  totally  occluded  sensor,  and,  as 
mentioned  above,  R{d)  =  0  at  all  distances.  For  k  =  1,  no  occlusions 
exist,  and  we  get  the  basic  relationship.  All  other  values  of  k  reflect 
varying  levels  of  occlusion,  and  their  effect  is  to  reduce  the  probabil¬ 
ity  of  detection.  Figure  B.6  depicts  the  extreme  cases  and  two  that  are 
intermediate.  We  assume,  as  in  Figure  B.4,  that  dmin  =  0. 

Another  use  of  r(d)  might  be  to  assess  the  effectiveness  of  sensor 
tasking.  When  sensors  are  tasked  to  focus  on  a  particular  aspect  of 
the  battlespace,  the  practical  effect  is  to  reduce  the  distance  between 
the  sensor  and  the  target — thus  improving  the  reliability  of  the 
tasked  sensors.  For  example,  Sensor  1  in  Figure  B.4  has  a  probability 
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- No  occlusions  Partially  occluded 

- Severly  occluded  Totally  occluded 


Figure  B.6 — Accounting  for  Occlusions 


of  detection  of  0.368  at  1  km  from  the  target,  so  R[  1)  =  0.368.  Suppose 
this  is  the  closest  the  sensor  is  able  to  approach  the  target,  and  fur¬ 
ther  suppose  that  only  Sensor  1  types  are  available.  If  we  task  three  of 
these  to  observe  the  target,  the  detection  probability  becomes 

f?3(d)  =  l-U-0.368)3. 

For  a  Sensor  1  type,  this  is  equivalent  to  being  approximately  0.3  km 
from  the  target;  therefore,  f?3(0.3)  ~  0.748. 

Collective  Completeness 

The  better  the  sensor  suite  (in  terms  of  sensor  performance  and 
operational  integration),  the  more  likely  it  is  that  the  number  of  tar¬ 
gets  detected  will  be  the  total  in  the  AO.  Given  an  expression  for  the 
reliability  of  individual  sensors,  what  is  needed  is  an  assessment  of 
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Figure  B.7 — Multisensor  Operations 


the  integrated  sensor  suite.  Figure  B.7  depicts  three  possible  opera¬ 
tional  modes:  independent  operation,  cueing,  and  mixed  mode.3  The 
structure  of  the  network  is  exemplary.  The  fact  that  all  reports  end  at 
a  single  fusion  center  is  not  central  to  the  assessment.  Clearly,  fusion 
can  take  place  anywhere  in  the  ISR  and  fusion  system  and  may  be 
either  distributed  or  centralized.  Indeed,  one  value  of  measuring  the 
quality  of  information  generated  from  sensor  operations  and  fusion 
processes  is  to  evaluate  the  effectiveness  of  alternative  architectures. 

It  is  important  to  note  here  that  these  alternative  architectures  are 
not  to  be  confused  with  alternative  multisensor  decision  structures 
(see  Waltz  and  Llinas,  1990).  A  multisensor  decision  structure 
addresses  the  way  complementary  sensors  arrive  at  a  detection  deci¬ 
sion  before  transmitting  the  decision  to  the  fusion  center.  In  a  cen¬ 
tralized  decision  structure,  each  sensor  passes  its  observations 
directly  to  an  observation  facility  within  the  fusion  center,  where  a 
combined  decision  is  taken.  In  a  decentralized  structure,  each  sensor 


3This  may  not  be  an  exhaustive  set.  For  example,  a  standby  configuration  is  also  pos¬ 
sible,  in  which  a  less-reliable  sensor  is  held  in  reserve  to  temporarily  replace  a  more- 
reliable  system  that  has  failed.  See,  for  example,  Dhillon  (1988). 
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renders  a  decision,  which  is  forwarded  to  the  fusion  center.  The 
architecture  in  Figure  B.7  can  support  either  structure. 

Independent  Operation.  This  is  the  simplest  form  of  operation.  Each 
sensor  observes  the  AO  independently  and  reports  its  detections  to 
the  common  fusion  center.  This  is  essentially  parallel  operation  in 
reliability  analysis,  and  it  increases  the  likelihood  that  a  detection(s) 
will  occur.  To  calculate  the  sensor  suite  reliability  for  this  case,  we 
used 


i?(d)  =  1-  1  -R^d)  1  -R2(d)  1  -R3(d) 


Cueing.  In  cueing  operations,  one  sensor  detects  a  target  and  notifies 
another  to  confirm  the  detection  (or,  in  practice,  to  provide  addi¬ 
tional  data  on  the  target).  A  report  is  rendered  when  the  two  sensors 
have  detected  the  target.  This  results  in  a  conditional  reliability, 
f?i|2(d),  that  is,  the  reliability  of  sensor  S ,  given  the  reliability  of 
sensor  S2.  In  this  formulation,  the  two  sensors  are  not  independent. 
That  is,  the  reliability  of  the  cued  sensor  depends  on  the  reliability  of 
the  cueing  sensor.  This  further  requires  that  a  failure  rate,  r^U),  be 
determined  for  each  cued-cueing  pair. 

Mixed  Mode.  This  is  the  most  likely  operational  mode,  a  mixture  of 
both  independent  operations  and  cueing.  The  overall  system  reliabil¬ 
ity  in  this  case  is  dependent  on  the  complexity  of  the  system  struc¬ 
ture.  For  the  simple  case  depicted  in  the  diagram,  this  is 


R(d)  =  1  -  [l  -  R3  (d)RA  (d)T  1  -  R2  (d)j  1  -  R3  (d) 


A  reasonable  estimate  for  the  number  of  targets  in  the  AO  detected 
by  the  sensor  suite  at  time  t  then  is  100 R[d).  If,  for  example,  10 
targets  are  detected,  and  R{d)  =  0.4,  the  estimated  total  number  of 
units  or  targets  in  the  AO  is  10/0.4  =  25.  The  poor  reliability  of  the 
sensor  system  in  this  example  is  a  function  both  of  the  system 
architecture  and  the  distances  the  sensors  are  to  the  targets. 

Assuming  the  fraction  of  the  AO  the  sensor  suite  covers  is  c2,  the  fol¬ 
lowing  estimates  the  completeness  transformation  metric  using  the 
suggested  reliability  model: 
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Qcom  (Ft  I  Fo)  =  «M(l-^C2) 


DISSEMINATION 

In  Chapter  Three,  we  suggested  that  one  way  to  assess  the  complete¬ 
ness  of  the  dissemination  process  is  to  assess  the  reliability  of  the 
dissemination  network.  This,  in  turn,  requires  that  we  assess  the  reli¬ 
ability  of  the  links  in  the  network.  In  this  section  of  the  appendix,  we 
illustrate  the  process  by  postulating  a  small  subnetwork  consisting  of 
a  single  fusion  center,  F;  three  CROP  users,  Uv  U2,  and  l/3;  and  two 
relay  nodes,  7\  and  T2  (transshipment  nodes,  in  the  language  of  net¬ 
work  theory).  Figure  B.8  illustrates  the  connectivity  among  the 
nodes.  We  assumed  that  the  network  is  cyclic — that  two-way  com¬ 
munication  is  possible  on  all  links.  However,  we  ruled  out  cycles  in 
communicating  between  nodes  and  assessed  the  likelihood  that  the 
three  user  nodes  are  connected.  Clearly,  this  depends  on  the  reliabil¬ 
ity  of  the  links  on  the  various  paths  from  the  fusion  center  to  the  user 
nodes. 

Using  a  reliability  model,  we  focus  on  link  (Lj)  reliability,  Tl^q),  where 
the  failure  rate  function,  r[q),  is  a  measure  of  communications  qual¬ 
ity  and,  as  such,  is  a  function  of  the  SNR,  jamming,  bandwidth,  etc. 


RAND  MR1467-B.8 


T 


Figure  B.8 — A  Communications  Subnetwork 
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Depending  on  the  convention  adopted,  we  can  have  R^q)  increase  as 
q  increases  or  the  opposite.  For  example,  if  r{q)  =  q, 

Ri[q]  =  e  2  , 

and  Rj{q)  decreases  with  increasing  q.  This  can  be  used  to  model  the 
effects  of  jamming,  for  example.  However,  in  the  case  of  SNR  and 
bandwidth,  we  would  expect  the  reliability  of  the  link  to  increase 
with  increasing  values  of  q.  To  get  this  opposite  relationship,  we  can 
set 


_q 

Ri[q)  =  \-e  2  . 

The  probability  that  a  user  is  connected  is  then  the  probability  that  at 
least  one  path  between  the  fusion  facility  and  the  user  is  available, 
given  the  values  of  q  for  each  of  the  links.  The  information  transmit¬ 
ted  over  the  network  is  then  the  joint  probability  that  all  the  users  are 
connected  at  time  t.  For  the  simple  network  in  Figure  B.8,  we  com¬ 
pleted  the  calculations  depicted  in  Table  B.4.  The  last  column  is  the 
probability  that  the  individual  user  is  connected;  therefore,  the 


Table  B.4 

Network  Completeness  Assessment 
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Path 

Path  Reliability 

Probability  Connected, 
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1  1 
jo  jo 
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5  T 
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jo 

U2 

l7 
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u3 
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4  4  4 
l5  4  4  4 

Rs(q)  R,  (q)  R3( q) 

R7i q)  R3( q)  R2iq) 
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[1  -R3R6R,R2] 
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assessment  of  complete  network  connectivity  is  the  probability  that 
all  users  are  connected: 


at = rili  &■ 


INDIVIDUAL  SITUATIONAL  AWARENESS 

In  Chapter  Four,  we  discussed  the  relationship  between  the  aware¬ 
ness  of  the  alternative  decision  agents  and  each  of  the  total  informa¬ 
tion  quality  measures.  We  used  AG  [0,1]  to  denote  individual 
awareness.  This  section  illustrates  two  possible  functions  relating 
information  completeness  to  the  awareness  that  decision  agents  with 
varying  capabilities,  denoted  4>„  can  achieve.  For  both  functions,  we 
generated  a  set  of  isocurves  that  vary  with  4>,-.  Accordingly,  we  modi¬ 
fied  the  awareness  metric  to  Aj,(. 

The  first  is  a  series  of  exponential  functions  of  the  form  A<s>i  =  1-  e&  c 
The  shape  of  the  curve  depends  on  the  parameter  /3;  the  parameter’s 
value  is  chosen  to  reflect  the  decision  agent’s  characterization.  For 
example,  the  curve  is  shifted  to  the  right  to  reflect  the  “unaware” 
decision  agent’s  (<J>4)  requirement  for  high  degrees  of  completeness. 
Figure  B.9  illustrates  the  four  curves  discussed  in  the  text.4 

The  so-called  logistic  or  S-curve  may  also  describe  a  decision  agent’s 
variable  behavior.5  That  is,  for  low  levels  of  information  quality,  his 
awareness  is  at  its  lowest  level.  For  some  region  of  awareness  above 
this  threshold,  his  awareness  increases  rapidly,  tapering  off  consid¬ 
erably  beyond  this  region.  Figure  B.10  illustrates  this  behavior.  These 
curves  are  all  of  the  form 


A 


^Po+Pi(‘l>i)c 
l  +  gPo+Pl(<1>i)C 


B.l 


4This  figure  also  appears  in  the  main  text  as  Figure  4.2. 

5This  curve  is  sometimes  referred  to  as  the  logistics  response  function  or  the  growth 
curve  (see  Neter  and  Wasserman,  1974). 
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Figure  B.9 — The  Effect  of  Complete  Information 
on  Situational  Awareness 
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Figure  B.10 — S-Curve  Representation  of  Completeness 
on  Situational  Awareness 
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The  parameters  /30  and  /3(<t>j)  reflect  the  decision  agent’s  characteri¬ 
zation  and  may  be  determined  through  analysis  of  variance  tech¬ 
niques. 

The  relationship  defined  in  equation  B.l  is  more  generally  applicable 
to  quality  of  information.  Extending  equation  B.l,  the  general  rela¬ 
tionship,  yields 


Po+Pi^ijQ^O’h’hih) 

A*,  = - 7 \ — 7 - r  ,  B.2 

'  l  +  /o+Pi(®i)Q(Fo.hT2,F3) 

where  Q(F0,F1,F2,F3)  is  the  quality  of  the  observed  CROP. 

In  both  formulations,  A<s>i  is  interpreted  to  be  the  degree  of  individual 
situational  awareness — the  fraction  of  the  observed  CROP  that  an 
individual  decisionmaker  with  ability  O,  is  able  to  realize.  It  is  para¬ 
metrically  related  to  O,,  in  that  we  postulate  a  finite  set  of  capabilities 
fh  (<!>,• )  and  adjust  accordingly. 

From  Figure  B.9,  the  degree  of  individual  situational  awareness  that 
the  four  decision  agent  capability  levels  plq,  4>2,  «&3,  and  3q)  attain  for 
a  CROP  with  quality  Q(F0,F1,F2,F3)  =  0.4  is  approximately  100,  88,  35, 
and  25  percent  of  the  fused  feature  vectors  in  the  CROP,  respectively. 
For  Figure  B.10,  it  is  85,  52,  25,  and  17  percent  of  the  fused  feature 
vectors,  respectively. 

Transactive  Memory  System  Model 

A  transactive  memory  system,  A,  is  a  set  of  individual  memory  sys¬ 
tems  in  combination  with  the  communication  that  takes  place 
between  individuals.  The  transactive  memory  system  model 
addresses  the  storage  and  movement  of  information  among  actors. 
In  Chapter  Four,  we  discussed  a  general  construct  for  measuring 
what  we  referred  to  as  “team  hardness”  in  such  a  system.  This  sec¬ 
tion  suggests  a  specific  model  that  possesses  the  appropriate  charac¬ 
teristics. 

A  basic  assumption  of  the  model  is  that  the  rate  at  which  the  ratio  of 
shared  information  storage  or  complexity  in  the  transactive  memory 
grows  is  linear.  The  rate  depends  on  the  degree  of  consistency  of 
team  membership  over  time  and  on  the  complexity  of  the  situation. 
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We  represented  the  rate  at  which  hardness  increases  as  a  parameter, 
k,  expressed  in  units  of  hardness  per  unit  time.  With  a  constant 
growth  rate  model,  a  simple  increasing  exponential  can  represent 
team  hardness.  If  we  let  t  represent  the  time  elapsed  since  the  start  of 
the  operation  and  rthe  length  of  time  the  team  has  been  training  or 
operating  together,  transactive  memory  is  TM[T)  =  1-  e  kl ,  where  T  = 
x  +  t .  The  operation  is  assumed  to  have  begun  at  time  t  -  0.  Figure 
B.  1 1  depicts  the  behavior  of  TM{T)  for  various  values  of  k. 

Teams  with  high  team  hardness  rates  (/<.',)  rapidly  develop  the  shared 
information  and  transactive  memory  system  complexity  necessary  to 
respond  to  a  given  situation.  Teams  with  moderate  team  hardness 
rates  (fc3)  must  refine  a  transactive  memory  system  while  responding 
to  the  current  situation.  Teams  with  a  low  team  hardness  rate  (kA) 
must  establish  and  develop  the  transactive  memory  system  while 
responding  to  the  current  situation. 


RANDMR1467-B.il 


Figure  B.  1 1 — Transactive  Memory  for  Alternative 
Values  of  Team  Hardness 
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A  Revised  Model  of  Situational  Awareness 

The  next  logical  step  is  a  revised  measure  of  individual  situational 
awareness  that  combines  the  quality  of  the  observed  CROP  and  the 
effects  of  the  transactive  memory  system,  A^it).  This  measure  is  time 
dependent  because  of  the  introduction  of  transactive  memory.  It  is 
now  interpreted  to  be  the  fraction  of  the  observed  feature  vectors 
that  the  individual  decisionmaker  with  capability  <&,  and  with  benefit 
of  team  participation  can  realize  when  the  operation  has  been  in 
progress  for  t  time  units. 


ePo+Pi(®i)[Q(Fo.Ii.F2.F3)+™(r)] 
l  +  elh+Pi(®I)[d(F°'h.F2.F3)+™(r)]  ' 

Note  that  if  /?  — »■  oo  at  time  t  =  0,6  Q[0(f)]  =  0  and  TM{T)  =  TM{ x)  =  0  for 
a  team  with  no  collective  experience. 

A  MODEL  OF  SHARED  SITUATIONAL  AWARENESS 

As  Chapter  Four  suggested,  the  next  step  is  to  evaluate  the  situational 
awareness  of  the  team  when  it  is  working  together,  SA{t),  recognizing 
that  this  measure  will  also  will  be  time  dependent.  This  was  referred 
to  as  shared  situational  awareness.  The  general  relationship  is 
expressed  as 


SA(t)-g(A^,  F5). 


the  collective  fraction  of  the  observed  CROP,  Q(F0,F1,F2,F3),  that  the 
entire  team  can  interpret.  In  this  formulation,  F5  represents  the 
observed  CROP  agents  of  capability  fl>,at  time  t  can  realize.  What 
follows  is  a  suggested  methodology  for  calculating  SA(f). 

Suppose  we  let  m  be  the  number  of  feature  vectors  in  the  observed 
CROP,  i.e.,  the  cardinality  of  the  set  F5  is  m ,  or  ||F5||  =  m,  and  the 
number  of  feature  vectors  realized  by  all  team  members  with 
capability  <f>,  at  time  L  is 


6In  practice,  a  value  of  -5  is  sufficient. 
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However,  the  feature  vectors  interpreted  may  not  be  the  same  for 
each  individual.  For  team  member  j  with  capability  <1>„  the  cardinal¬ 
ity  of  the  set  of  feature  vectors  an  individual  can  interpret  is  therefore 


Since  it  is  impossible  to  know  which  vectors  are  realized,  we  instead 
examined  the  possible  overlaps.  The  intersection  set  between  two 
team  members,  j  and  k,  one  with  capability  ,  and  one  with  capabil¬ 
ity  4>;  is  given  by 


F/vi)  ,■  f'1  Pfco,  • 


The  smallest  number  of  elements  in  this  set  (overlap)  is 

and  the  largest  it  can  be  is 

min | mA'^ . (f ), mA'<l>k (f  )j  . 

Therefore,  a  reasonable  estimate  of  the  fraction  of  overlapping  fea¬ 
ture  vectors  that  team  members  j  and  k  with  capabilities  <1>,  and  <&/ 
realize  is  the  average  of  these  two  quantities: 


G 


jk 


A^,k[t  ||  +  min 


21 

=  imaxMi If)) . 


B.3 


Gik(t)  =  (i/2)A;l>i(t). 


For  example,  if  i  =  l, 
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This  appears  to  be  right,  in  that,  on  average,  two  team  members  that 
realize  the  same  number  of  feature  vectors  will,  on  average,  have  half 
in  common.  Equation  B.3  therefore  represents  the  degree  of  consen¬ 
sus  between  two  team  members. 

Next,  we  needed  to  account  for  the  composition  of  the  team  itself, 
that  is,  for  the  number  of  each  capability  type  present  in  the  team 
and  the  size  of  the  team.  We  might  have  done  this  by  assessing  the 
degree  of  consensus  among  all  pairwise  combinations  of  team  mem¬ 
bers.  We  paired  all  possible  capability  types,  making  the  calculation 
above  and  averaging  the  result,  or,  for  a  team  of  size  n: 

SA(t)  =  l/^tirH+iGij(t).  BA 

The  degree  of  shared  situational  awareness,  SA{t),  is  the  collection  of 
fused  feature  vectors  at  time  t  that  members  of  a  team  realize  simi¬ 
larly  if  they  collaborate. 

Equation  B.4  approximates  the  degree  of  shared  situational  aware¬ 
ness  because  our  measure  for  the  degree  of  consensus  is  an  average. 
We  also  assumed  that  all  pairwise  combinations  of  collaborating 
individuals  have  the  average  measure  for  degree  of  consensus. 
Despite  these  limitations,  equation  B.5  does  tie  together  the  individ¬ 
ual  and  team  components  of  shared  situational  awareness.  We 
acknowledge  that  more-sophisticated  relationships  may  be  devel¬ 
oped  but  believe  that  the  equation  captures  the  essential  compo¬ 
nents  necessary  for  further  research  into  shared  situational  aware¬ 
ness. 


Appendix  C 

SPREADSHEET  MODEL 


There  is  no  study  that  is  not  capable  of  delighting  us  after  a 
little  application  to  it. 

— Alexander  Pope 


We  designed  measures  and  metrics  the  main  report  describes  and 
Appendix  B  details  to  reflect  how  the  quality  of  the  information  pro¬ 
duced  within  the  three  C4ISR  domains  and  during  team  collaboration 
affects  shared  situational  awareness.  The  assumption  throughout 
has  been  that  the  process  that  produces  the  information  (sensors, 
fusion  facilities  and  processes,  distribution  networks)  is  not  neces¬ 
sarily  known,  and  we  are  not  concerned  about  how  the  information 
is  transformed  as  it  moves  through  the  information  value  chain.  The 
quality  of  the  processes  is  expressed  through  the  quality  metrics  in 
the  form  of  parameters  and  alternative  probability  functions.  The 
quality  of  team  interactions  is  assessed  by  examining  the  capabilities 
of  individual  team  members,  the  degree  of  team  hardness,  and  the 
level  of  collaboration. 

This  appendix  illustrates  the  assessment  process  through  a  simple 
spreadsheet  model  that  allows  us  to  alter  quality  parameters.  The 
reliability  curves  are  taken  to  be  constant,  except  that  the  quality 
parameters  can  be  varied. 
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MODELING  INFORMATION  QUALITY 

A  simple  spreadsheet  model  was  developed  to  illustrate  the  informa¬ 
tion  quality  assessment  process.1  Only  the  completeness  measures 
are  included  in  the  version  described  below.  However,  the  model  can 
easily  be  expanded  to  include  correctness  and  currency.  Time,  t;  the 
distance  between  the  target  and  the  sensor,  d;  and  the  level  of  jam¬ 
ming,  q,  are  independent  variables.  A  three-dimensional  response 
surface  (the  composite  completeness)  is  displayed  using  time  on  one 
axis  and  either  distance  or  the  level  of  jamming  on  the  other.  The 
independent  variable  not  used  is  treated  as  a  parameter. 

Sensors 

We  modeled  three  sensors,  each  with  a  unique  reliability  function 
(see  Table  C.l).  In  all  cases,  we  took  the  minimum  distance  for 
detection  to  be  0.  The  occlusion  settings,  klt  varied,  as  did  the  dis¬ 
tance  between  the  sensor  and  the  target. 

Table  C.l 

Sensor  Quality  Functions 


Sensor 

r{s) 

Ri(d) 

St 

1 

kxe~d 

s2 

s 

d2 

k2e^ 

S3 

*2 

d3 

k?e  3 

S1IS3 

0.5 

k4e“°'5d 

St|S2,S3 

0.333 

7.  -0.333d 

/e5e 

1The  model  is  titled  Infoview  and  was  designed  and  implemented  by  RAND  colleague 
Thomas  Sullivan. 
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We  examined  three  architectures:  independent  operation,  queuing, 
and  a  mixture  of  both.  Figure  C.  1  describes  all  three.  The  system  reli¬ 
abilities  for  all  three  are  as  follows: 


a.  independent  operations: 

R(d)  =  1  -  1  -  Rx  (d)  1  -  R2  (d)  1  -  f?3  (d) 


=  1- 


1  -kxe 


1  -k2e  2 


1  -he  3 


b.  queued  operations: 

R(d)  =  Rm(d)  =  k5e-0333d  . 

c.  mixed  operations: 


d2 

1  -  Ryp  (d,  j 

1— 1 

1 

to 

s 

=  1- 

1  -k4e~°-5d 

l-k2e  2 

If  we  further  assumed  that  the  sensors  for  all  three  architectures  cov¬ 
ered  the  same  portion  of  the  area,  c2  =  0.8,  yielding  the  following 
completeness  metric:  QomCF^Fo)  =  R{d){  1  -e-0-8). 

Fusion 

We  modeled  a  single  fusion  facility  consisting  of  IMINT  and  ELINT 
processing  centers  and  a  central  processing  center,  as  depicted  in 
Figure  C.2.  Each  of  the  three  facilities  was  modeled  using  the  metric 
suggested  in  Chapter  Three: 

c  =  a  +  a|l-e“af  j . 

Recall  that  a  represents  the  fraction  of  the  detected  targets  that  the 
sensors  themselves  can  fuse  and  that  0<n  +  a<lis  the  maximum 
fraction  of  the  detections  capable  of  being  classified  at  the  fusion 
center.  The  third  parameter,  a,  is  the  rate  at  which  detections  are 
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Figure  C.l — Alternative  Sensor  Architectures 
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Figure  C.2 — Model  Fusion  Architecture 
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classified.  These  parameters  are  set  in  the  model  as  shown  in  Figure 
C.3.  From  this  the  fusion  architecture,  we  can  calculate  completeness 
as  follows: 


C  -  [1  -  (1  -  CIMINT)[1  -  CpjjNT )]  Cc  , 

where  cc  represents  the  completeness  of  the  central  fusion  facility. 

Network 

The  network  modeled  is  the  sample  network  in  Figure  B.8.  For  all 
links  in  the  network,  we  modeled  the  completeness  of  the  messages 
transmitted  in  terms  of  the  level  of  jamming.  In  addition,  we 
assumed  that  the  effects  would  be  the  same  for  all  links  and  that  they 
are  defined  by  the  failure  rate  function  r{q)  =  q.  Therefore,  the  com¬ 
pleteness  metric  for  each  link  is  as  follows: 

RL(q)  =  e  2  . 

Applying  this  to  the  network  results  in  a  network  completeness  met¬ 
ric,  N,  calculated  using  the  procedures  presented  in  Table  B.3. 
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Figure  C.3 — Infoview  Information  Domain  GUI 
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MODELING  TEAM  DECISIONMAKING 

We  expanded  the  spreadsheet  model  above  to  account  for  the  next 
step  in  the  process.  The  quality  of  information  produced  thus  far 
becomes  one  factor  in  the  overall  assessment  of  situation  awareness. 
Typically,  the  parameter  settings  established  to  produce  a  given  level 
of  information  quality  in  the  information  domain  are  “carried  over” 
into  the  cognitive  domain. 

The  additional  parameters  needed  to  complete  the  assessment  of 
shared  situational  awareness  are  the  degree  of  team  hardness  and 
the  number  of  team  participants  possessing  the  different  levels  of 
capability.  The  equation  used  to  produce  the  level  of  transactive 
memory  achieved  depends  on  the  parameter  selected  for  team  hard¬ 
ness.  In  this  case  we  selected  a  value  of  k  =  1.0,  leading  to 

TM{T)  =  1  -elor. 

We  must  now  assess  the  effect  of  the  level  of  individual  situational 
awareness  that  the  members  of  the  team  with  capability  O,  possesses 
for  each  capability  level.  For  this  example,  we  assumed  four  levels  of 
capability,  and  therefore 

e-5+fi1(oi)[Q(F0,F1,F2,F3)+rM(r)] 


represents  the  fraction  of  the  observed  feature  vectors  that  the  indi¬ 
vidual  decisionmaker  with  capability  4>,  and  with  benefit  of  team 
participation  can  interpret.  For  this  example,  we  assumed  that  there 
are  nine  team  members:  two  with  capability  three  with  capability 
i>2>  one  with  capability  <h3,  and  three  with  capability  4>4  Note  also 
that  we  set  ft  =  -5.  The  coefficient  ft  is  referred  to  as  the  threshold 
and  is  set  at  0.15  for  this  example. 

It  is  not  necessary  that  we  know  the  number  of  feature  vectors  in  the 
observed  CROP,  since  we  are  only  concerned  with  the  average  frac¬ 
tion  of  overlapping  features: 

Gjk(t)  =  (*)} 

for  every  pair  of  participants  in  the  decisionmaking  team,  j,  k. 


-5+p1(oi)[Q(F0,F1,F2,F3)+rM(r)' 
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With  this,  the  total  shared  situational  awareness  is  calculated  to  be: 

THE  GRAPHICAL  USER  INTERFACES 

Two  graphical  user  interfaces  (GUIs)  are  used  to  assess  shared  situa¬ 
tional  awareness.  The  first  depicts  information  domain  effects  on 
information  quality,  and  the  second  depicts  the  effects  of  informa¬ 
tion  quality  and  cognitive  domain  conditions  on  shared  situational 
awareness. 

Information  Domain  Assessments 

Figure  C.3  depicts  the  GUI  for  the  completeness  model  only.  The 
slide  bars  to  the  left  of  the  figure  allow  the  user  to  select  the  desired 
parameter  settings.  Beginning  with  the  sensor  subdomain,  the  user 
selects  the  level  of  occlusion  and  the  sensor  architecture.  The  two 
INT  disciplines,  IMINT  and  ELINT,  and  the  central  processing  facility 
are  displayed  next,  each  with  three  parameter  sliders.  The  first,  a, 
determines  the  rate  at  which  the  observations  are  classified;  the  sec¬ 
ond  parameter,  a,  represents  the  fraction  of  the  detected  targets  that 
can  be  fused  by  the  sensors  themselves;  and  the  third  parameter,  a, 
is  selected  such  that  a  +  a  is  the  upper  bound  on  the  fraction  of 
detections  that  can  be  classified.  Recall  that  combinations  of 
automation  and  control  protocols  can  affect  the  classification  rate  at 
each  of  the  fusion  facilities.  The  effects  are  therefore  reflected  in  the 
values  of  the  parameters  a,  a,  and  a.  The  sole  network  parameter,  q, 
is  the  level  of  jamming  on  all  links.  If  the  “Distance  vs.  Time"  button 
is  selected  from  the  “Plot"  options  under  the  graph,  q  is  defined 
parametrically,  since  d  is  the  selected  independent  variable.  The 
opposite  is  true  if  “Jamming  Parameter  vs.  Time”  is  selected. 

The  three-dimensional  figure  to  the  right  represents  the  degree  of 
completeness  for  a  given  sensor  distance  and  processing  time.  The 
decline  along  the  distance  axis  for  a  given  processing  time  reflects 
the  fact  that  completeness  deteriorates  as  the  distance  from  the  sen¬ 
sor  to  the  target  increases.  The  behavior  along  the  time  axis  for  a 
given  distance  reflects  the  fact  that  completeness  is  likely  to  increase 
with  more  time  available  to  process  information. 
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Finally,  note  the  selection  bar  titled  “Shared  Awareness  for  these  set¬ 
tings”  at  lower  left.  The  effects  of  information  quality  and  the  other 
factors  on  shared  awareness  are  evaluated  when  this  button  is 
selected.  This  button  sets  the  completeness  quality  level  that  is  input 
to  the  evaluation  of  shared  awareness.  Selecting  this  option  will  take 
the  user  to  another  GUI  that  will  be  discussed  next. 

The  three-dimensional  surface  to  the  right  in  Figure  C.3  results  from 
the  settings  selected  at  the  left.  In  this  case,  the  maximum  complete¬ 
ness  is  approximately  60  percent  and  occurs  at  approximately  2  time 
units,  when  the  sensors  are  roughly  between  0  and  1  distance  unit 
from  the  target.  Note  that  completeness  falls  off  as  the  distance  from 
the  target  increases.  Note  also  that  completeness  rises  rather  rapidly 
with  time.  This  is  due  to  the  shape  of  the  fusion  curves  and  thus 
reflects  the  degree  of  control  and  automation. 

Cognitive  Domain  Assessments 

Figure  C.4  depicts  the  GUI  for  evaluating  shared  situational  aware¬ 
ness.  As  with  the  information  domain  GUI,  the  slider  bars  at  the  left 
allow  various  parameter  settings.  The  first  bar  sets  the  value  of  ft  for 
all  decision  agents — 0.15  for  this  example.  The  second  bar  estab¬ 
lishes  the  value  for  team  hardness  (1.8  in  this  case).  The  next  four 
bars  set  the  number  of  decision  agents  at  each  of  the  four  capability 
levels  participating  in  the  decision.  The  settings  depicted  are  consis¬ 
tent  with  the  numbers  stated  above.  Note  also  that  ft  =  -5,  as  sug¬ 
gested  above. 

The  smaller  box  at  the  lower  right  records  the  values  of  the  parame¬ 
ters  set  in  the  information  domain  GUI.  These  settings  are  used  to 
calculate  the  information  quality  at  the  various  times,  t,  during  the 
decisionmaking  process.  Information  domain  settings  are  constant 
for  all  parameter  settings  in  the  cognitive  domain  GUI. 

Finally,  the  three-dimensional  figure  depicted  represents  the  degree 
of  shared  situational  awareness,  SA(t),  for  varying  values  of  t  and  r, 
the  time  the  decisionmaking  process  has  been  in  progress  and  the 
experience  of  the  team  in  terms  of  the  number  of  months  it  has 
worked  as  a  team,  respectively. 

Note  that  teams  that  have  been  operating  together  less  than  approx¬ 
imately  two  weeks  have  very  little  situational  awareness,  regardless 
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Figure  C.4 — Infoview  Cognitive  Domain  GUI 


of  how  long  the  operation  extends.  Also  note  that  shared  situational 
awareness  increases  continually  in  both  directions,  as  the  team  gains 
training  and  experience  from  the  ongoing  operation.  However,  the 
rate  of  increase  tapers  dramatically  at  around  6  weeks  of  training  and 
about  1  minute  of  operational  experience.  This  is  primarily  due  to 
the  behavior  of  information  quality. 

ANALYSIS 

The  spreadsheet  model  can  be  used  to  conduct  analysis.  A  basic 
assumption  for  this  version  of  the  model  is  that  the  mathematical 
representations  for  completeness  in  all  domains  accurately  reflect 
experience  with  the  components  of  each  domain  and  that  the  math¬ 
ematical  representations  for  transactive  memory  and  individual  sit¬ 
uational  awareness  adequately  reflect  their  effects  on  shared  situa¬ 
tional  awareness.  These  assumptions  remain  to  be  tested. 

Typically,  a  research  question  will  focus  on  the  effects  of  one  or  more 
of  the  model  parameters  on  information  quality  and,  for  a  fixed 
decisionmaking  team,  the  effect  on  shared  situational  awareness.  For 
example,  we  may  wish  to  examine  the  effects  of  jamming  or  the  level 
of  automation  and  sensor  control  on  completeness  given  a  deci¬ 
sionmaking  team  composed  as  in  the  example  above.  The  other 
parameters  may  remain  fixed  at  some  appropriate  level  or  may  take 
on  a  range  of  settings.  All  this  raises  the  issue  of  combinatorial 
analysis.  Since  each  of  the  slider  bars  in  the  model  is  continuous,  an 
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infinite  number  of  combinations  are  possible.  Consequently,  some 
bounds  must  be  placed  on  the  selections. 

Suppose  the  research  question  were:  “How  does  jamming  affect 
completeness  and  what  is  its  subsequent  effect  on  shared  situational 
awareness?”  We  would  further  suppose  the  following:  (1)  that  two 
sensor  architectures  are  of  interest:  mixed  and  independent;  (2)  that 
there  is  very  little  automation  and  sensor  control  in  IMINT  fusion 
( a -  0.2  and  a  -  0.5);  (3)  that  there  is  considerable  automation  but 
little  control  in  ELINT  fusion  ( a  -  0.2  and  a  =  0.95);  (4)  that  there  is 
considerable  automation  and  control  at  the  central  processing 
facility  ( o =  1  and  a  =  0.95);  (5)  that  there  is  no  “onboard”  fusion  at 
the  sensors  (for  all  facilities);  and  (6)  there  are  no  occlusions  (A:  =  1  for 
all  sensors) . 

Since  we  are  examining  the  effects  of  jamming,  we  must  also  specify 
the  range  of  the  sensors  to  the  targets.  Suppose  we  are  content  to  set 
three  ranges:  near  (0.2  km),  medium  (1.2  km),  and  distant  (2  km). 
Since  the  fusion  facility  parameters  are  fixed,  as  is  the  level  of  occlu¬ 
sions,  the  total  number  of  cases  to  be  examined  is  six  (two  architec¬ 
tures  and  three  ranges).  Figures  C.5  and  C.6  summarize  the  results. 

Now  let  us  extend  the  analysis  to  the  cognitive  domain.  Clearly,  for 
each  of  the  six  cases,  several  additional  cases  might  be  postulated 
simply  by  assuming  different  team  hardness  parameters  and  thresh¬ 
old  values  and  by  varying  the  competence  of  the  team  members.  For 
simplicity,  however,  we  will  hold  these  fixed  to  the  levels  discussed 
above.  Examining  only  the  independent  architecture  cases  depicted 
in  Figure  C.6  yields  the  three  shared  situational  awareness  depictions 
in  Figure  C.7. 

RESULTS 

It  is  clear  that,  regardless  of  the  level  of  jamming,  the  completeness  is 
never  more  than  about  55  percent  for  the  best  case  and  is  only 
slightly  better  for  the  independent  architecture  cases.  As  expected, 
the  farther  the  sensor  suite  is  from  the  target,  the  lower  the  system 
completeness.  In  all  cases,  as  jamming  increases,  completeness  falls 
off  rapidly. 

Note  also  that  shared  situational  awareness  in  the  independent 
architecture  case  is  sensitive  to  the  sensor  distances.  However,  for 
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1.  cf=  0.2 
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Figure  C.5 — Mixed  Architecture  Cases 


short  distances  (0.2  and  1.2  km),  the  differences  are  minimal 
(approximately  0.8  and  0.76  at  peak  performance  for  the  two  cases, 
respectively).  However,  as  the  distance  extends  to  2  km,  shared  sit¬ 
uational  awareness  drops  off  dramatically  (to  approximately  0.13), 
suggesting  that  additional  ranges  between  1.2  and  2.0  km  should  be 
investigated. 

A  result  such  as  this  would  suggest  that  the  research  question  was 
wrong.  Although  jamming  does  have  an  effect  on  system  complete¬ 
ness,  it  appears  that  the  levels  of  automation  and  sensor  control  at 
the  fusion  facilities  are  much  more  crucial.  Consequently,  it  might 
have  been  more  productive  to  ask  how  the  level  of  automation  and 
sensor  control  at  the  fusion  facilities  affects  information  complete¬ 
ness  and  shared  situational  awareness. 
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4.  d  =  0.2 
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Figure  C.6 — Independent  Architecture  Cases 
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Fixed  Inputs  from  Completeness  Calculations 


Figure  C.7 — Independent  Architecture  Shared  Situational  Awareness 
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